PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold (was: Re: [PATCH] Document GNU_PROPERTY_X86_ISA_1_[USED|NEEDED])

PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold (was: Re: [PATCH] Document GNU_PROPERTY_X86_ISA_1_[USED|NEEDED])

[Florian Weimer]

On 08/07/2018 10:41 PM, H.J. Lu wrote:
> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
> added to Linux Extensions to gABI:
> 
> https://github.com/hjl-tools/linux-abi
> 
> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
> processor-specific program property types for i386 and x86-64.

The specification is incomplete as far as alignment matters are concerned.

Is the link editor supposed to maintain separate segments for notes with 
different alignments?  Or is it possible to merge the notes into a 
single segment, potentially after adjusting alignment?

Is the link editor *required* to produce 8-byte alignment for notes in 
ELFCLASS64 objects?

Currently, we do not have agreement between binutils (particularly gold) 
and the glibc dynamic loader when it comes to alignment of PT_NOTE 
segments.  glibc will disregard property notes in ELFCLASS64 objects 
which have 4-byte alignment, but gold produces such notes.  This needs 
to be fixed.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold (was: Re: [PATCH] Document GNU_PROPERTY_X86_ISA_1_[USED|NEEDED])

[H.J. Lu]

On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>
>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>> added to Linux Extensions to gABI:
>>
>> https://github.com/hjl-tools/linux-abi
>>
>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>> processor-specific program property types for i386 and x86-64.
>
>
> The specification is incomplete as far as alignment matters are concerned.

https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf

has

2.1.7 Alignment of Note Sections

All entries in a PT_NOTE segment have the same alignment which equals to the
p_align field in program header.
According to gABI, each note entry should be aligned to 4 bytes in 32-bit
objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
aligned
to 4 bytes in both 32-bit and 64-bit objects. Note parser should use p_align for
note alignment, instead of assuming alignment based on ELF file class.


> Is the link editor supposed to maintain separate segments for notes with
> different alignments?  Or is it possible to merge the notes into a single
> segment, potentially after adjusting alignment?
>

It is possible.  We just need to place 4-byte aligned notes after 8-byte
aligned notes.

> Is the link editor *required* to produce 8-byte alignment for notes in
> ELFCLASS64 objects?

It is decided by the alignment of NOTE section, not by linker.

> Currently, we do not have agreement between binutils (particularly gold) and
> the glibc dynamic loader when it comes to alignment of PT_NOTE segments.
> glibc will disregard property notes in ELFCLASS64 objects which have 4-byte
> alignment, but gold produces such notes.  This needs to be fixed.

I don't believe this is true.  See above.

After this commit:

commit 8d81ce0c6d6ca923571e8b2bac132929f9a02973
Author: H.J. Lu <hjl.tools@gmail.com>
Date:   Tue Nov 28 09:56:47 2017 -0800

    Properly compute offsets of note descriptor and next note [BZ #22370]

    A note header has 3 4-bytes fields, followed by note name and note
    descriptor.  According to gABI, in a note entry, the note name field,
    not note name size, is padded for the note descriptor.  And the note
    descriptor field, not note descriptor size, is padded for the next
    note entry.  Notes are aligned to 4 bytes in 32-bit objects and 8 bytes
    in 64-bit objects.

    For all GNU notes, the name is "GNU" which is 4 bytes.  They have the
    same format in the first 16 bytes in both 32-bit and 64-bit objects.
    They differ by note descriptor size and note type.  So far, .note.ABI-tag
    and .note.gnu.build-id notes are always aligned to 4 bytes.  The exsting
    codes compute the note size by aligning the note name size and note
    descriptor size to 4 bytes.  It happens to produce the same value as
    the actual note size by luck since the name size is 4 and offset of the
    note descriptor is 16.  But it will produce the wrong size when note
    alignment is 8 bytes in 64-bit objects.

    This patch defines ELF_NOTE_DESC_OFFSET and ELF_NOTE_NEXT_OFFSET to
    properly compute offsets of note descriptor and next note.  It uses
    alignment of PT_NOTE segment to support both 4-byte and 8-byte note
    alignments in 64-bit objects.  To handle PT_NOTE segments with
    incorrect alignment, which may lead to an infinite loop, if segment
    alignment is less than 4, we treate alignment as 4 bytes since some
    note segments have 0 or 1 byte alignment.

glibc can handle both 4 byte and 8 byte NOTE alignments.


-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold (was: Re: [PATCH] Document GNU_PROPERTY_X86_ISA_1_[USED|NEEDED])

[H.J. Lu]

On Thu, Aug 16, 2018 at 6:19 AM, H.J. Lu <hjl.tools@gmail.com> wrote:
> On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>>
>>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>>> added to Linux Extensions to gABI:
>>>
>>> https://github.com/hjl-tools/linux-abi
>>>
>>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>>> processor-specific program property types for i386 and x86-64.
>>
>>
>> The specification is incomplete as far as alignment matters are concerned.
>
> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
>
> has
>
> 2.1.7 Alignment of Note Sections
>
> All entries in a PT_NOTE segment have the same alignment which equals to the
> p_align field in program header.
> According to gABI, each note entry should be aligned to 4 bytes in 32-bit
> objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
> Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
> aligned
> to 4 bytes in both 32-bit and 64-bit objects. Note parser should use p_align for
> note alignment, instead of assuming alignment based on ELF file class.
>
>
>> Is the link editor supposed to maintain separate segments for notes with
>> different alignments?  Or is it possible to merge the notes into a single
>> segment, potentially after adjusting alignment?
>>
>
> It is possible.  We just need to place 4-byte aligned notes after 8-byte
> aligned notes.

Wait.  We can't since the p_align field is used to compute the next note.
We need at least 2 NOTE segments where there are both 4-byte and
8-byte aligned notes.

>> Is the link editor *required* to produce 8-byte alignment for notes in
>> ELFCLASS64 objects?
>
> It is decided by the alignment of NOTE section, not by linker.
>
>> Currently, we do not have agreement between binutils (particularly gold) and
>> the glibc dynamic loader when it comes to alignment of PT_NOTE segments.
>> glibc will disregard property notes in ELFCLASS64 objects which have 4-byte
>> alignment, but gold produces such notes.  This needs to be fixed.
>
> I don't believe this is true.  See above.
>
> After this commit:
>
> commit 8d81ce0c6d6ca923571e8b2bac132929f9a02973
> Author: H.J. Lu <hjl.tools@gmail.com>
> Date:   Tue Nov 28 09:56:47 2017 -0800
>
>     Properly compute offsets of note descriptor and next note [BZ #22370]
>
>     A note header has 3 4-bytes fields, followed by note name and note
>     descriptor.  According to gABI, in a note entry, the note name field,
>     not note name size, is padded for the note descriptor.  And the note
>     descriptor field, not note descriptor size, is padded for the next
>     note entry.  Notes are aligned to 4 bytes in 32-bit objects and 8 bytes
>     in 64-bit objects.
>
>     For all GNU notes, the name is "GNU" which is 4 bytes.  They have the
>     same format in the first 16 bytes in both 32-bit and 64-bit objects.
>     They differ by note descriptor size and note type.  So far, .note.ABI-tag
>     and .note.gnu.build-id notes are always aligned to 4 bytes.  The exsting
>     codes compute the note size by aligning the note name size and note
>     descriptor size to 4 bytes.  It happens to produce the same value as
>     the actual note size by luck since the name size is 4 and offset of the
>     note descriptor is 16.  But it will produce the wrong size when note
>     alignment is 8 bytes in 64-bit objects.
>
>     This patch defines ELF_NOTE_DESC_OFFSET and ELF_NOTE_NEXT_OFFSET to
>     properly compute offsets of note descriptor and next note.  It uses
>     alignment of PT_NOTE segment to support both 4-byte and 8-byte note
>     alignments in 64-bit objects.  To handle PT_NOTE segments with
>     incorrect alignment, which may lead to an infinite loop, if segment
>     alignment is less than 4, we treate alignment as 4 bytes since some
>     note segments have 0 or 1 byte alignment.
>
> glibc can handle both 4 byte and 8 byte NOTE alignments.
>
>
> --
> H.J.



-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/16/2018 03:19 PM, H.J. Lu wrote:
> On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>>
>>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>>> added to Linux Extensions to gABI:
>>>
>>> https://github.com/hjl-tools/linux-abi
>>>
>>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>>> processor-specific program property types for i386 and x86-64.
>>
>>
>> The specification is incomplete as far as alignment matters are concerned.
> 
> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
> 
> has
> 
> 2.1.7 Alignment of Note Sections
> 
> All entries in a PT_NOTE segment have the same alignment which equals to the
> p_align field in program header.
> According to gABI, each note entry should be aligned to 4 bytes in 32-bit
> objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
> Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
> aligned
> to 4 bytes in both 32-bit and 64-bit objects. Note parser should use p_align for
> note alignment, instead of assuming alignment based on ELF file class.

This is still ambiguous, particularly based on your comments below.

>> Is the link editor supposed to maintain separate segments for notes with
>> different alignments?  Or is it possible to merge the notes into a single
>> segment, potentially after adjusting alignment?
>>
> 
> It is possible.  We just need to place 4-byte aligned notes after 8-byte
> aligned notes.

Based on section 2.1.7, this would not be valid by itself because the 
section needs to have 8-byte alignment (to satisfy the property notes 
requirement).  All notes in the segment need to have the same alignment 
(because p_align is supposed to be used for parsing).  So reordering 
alone will not produce a valid segment.

Part of the problem is that the note header is 12 bytes (not a multiple 
of 8), and that the name and descriptor lengths do not include the 
padding (which makes sense), so you really need a correct source of 
alignment.

If we want to generate a single segment (and I think we should), we need 
to realign the notes to a common alignment, either 4 or 8 bytes.  That's 
what gold seems todo right now, with 4-byte alignment.

>> Is the link editor *required* to produce 8-byte alignment for notes in
>> ELFCLASS64 objects?
> 
> It is decided by the alignment of NOTE section, not by linker.
> 
>> Currently, we do not have agreement between binutils (particularly gold) and
>> the glibc dynamic loader when it comes to alignment of PT_NOTE segments.
>> glibc will disregard property notes in ELFCLASS64 objects which have 4-byte
>> alignment, but gold produces such notes.  This needs to be fixed.
> 
> I don't believe this is true.  See above.

Which part?  I see the 4-byte segment alignment with gold from 
binutils-2.31.1-11.fc29.x86_64.

> After this commit:
> 
> commit 8d81ce0c6d6ca923571e8b2bac132929f9a02973
> Author: H.J. Lu <hjl.tools@gmail.com>
> Date:   Tue Nov 28 09:56:47 2017 -0800
> 
>      Properly compute offsets of note descriptor and next note [BZ #22370]
…
> glibc can handle both 4 byte and 8 byte NOTE alignments.

There's still this code in glibc, in sysdeps/x86/dl-prop.h:

   /* The NT_GNU_PROPERTY_TYPE_0 note must be aliged to 4 bytes in
      32-bit objects and to 8 bytes in 64-bit objects.  Skip notes
      with incorrect alignment.  */
   if (align != (__ELF_NATIVE_CLASS / 8))
     return;

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 6:31 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 08/16/2018 03:19 PM, H.J. Lu wrote:
>>
>> On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com>
>> wrote:
>>>
>>> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>>>
>>>>
>>>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>>>> added to Linux Extensions to gABI:
>>>>
>>>> https://github.com/hjl-tools/linux-abi
>>>>
>>>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>>>> processor-specific program property types for i386 and x86-64.
>>>
>>>
>>>
>>> The specification is incomplete as far as alignment matters are
>>> concerned.
>>
>>
>> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
>>
>> has
>>
>> 2.1.7 Alignment of Note Sections
>>
>> All entries in a PT_NOTE segment have the same alignment which equals to
>> the
>> p_align field in program header.
>> According to gABI, each note entry should be aligned to 4 bytes in 32-bit
>> objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
>> Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
>> aligned
>> to 4 bytes in both 32-bit and 64-bit objects. Note parser should use
>> p_align for
>> note alignment, instead of assuming alignment based on ELF file class.
>
>
> This is still ambiguous, particularly based on your comments below.

https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf

conforms to gABI unless stated otherwise.

>>> Is the link editor supposed to maintain separate segments for notes with
>>> different alignments?  Or is it possible to merge the notes into a single
>>> segment, potentially after adjusting alignment?
>>>
>>
>> It is possible.  We just need to place 4-byte aligned notes after 8-byte
>> aligned notes.
>
>
> Based on section 2.1.7, this would not be valid by itself because the
> section needs to have 8-byte alignment (to satisfy the property notes
> requirement).  All notes in the segment need to have the same alignment
> (because p_align is supposed to be used for parsing).  So reordering alone
> will not produce a valid segment.
>
> Part of the problem is that the note header is 12 bytes (not a multiple of
> 8), and that the name and descriptor lengths do not include the padding
> (which makes sense), so you really need a correct source of alignment.
>
> If we want to generate a single segment (and I think we should), we need to
> realign the notes to a common alignment, either 4 or 8 bytes.  That's what
> gold seems todo right now, with 4-byte alignment.

I was wrong.  We need 2 NOTE segments one fore 8-byte alignment and
one for 4-byte alignment.

>>> Is the link editor *required* to produce 8-byte alignment for notes in
>>> ELFCLASS64 objects?
>>
>>
>> It is decided by the alignment of NOTE section, not by linker.
>>
>>> Currently, we do not have agreement between binutils (particularly gold)
>>> and
>>> the glibc dynamic loader when it comes to alignment of PT_NOTE segments.
>>> glibc will disregard property notes in ELFCLASS64 objects which have
>>> 4-byte
>>> alignment, but gold produces such notes.  This needs to be fixed.
>>
>>
>> I don't believe this is true.  See above.
>
>
> Which part?  I see the 4-byte segment alignment with gold from
> binutils-2.31.1-11.fc29.x86_64.
>

glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
generates 4 byte alignment, it is a gold bug.

>> After this commit:
>>
>> commit 8d81ce0c6d6ca923571e8b2bac132929f9a02973
>> Author: H.J. Lu <hjl.tools@gmail.com>
>> Date:   Tue Nov 28 09:56:47 2017 -0800
>>
>>      Properly compute offsets of note descriptor and next note [BZ #22370]
>
> …
>>
>> glibc can handle both 4 byte and 8 byte NOTE alignments.
>
>
> There's still this code in glibc, in sysdeps/x86/dl-prop.h:
>
>   /* The NT_GNU_PROPERTY_TYPE_0 note must be aliged to 4 bytes in
>      32-bit objects and to 8 bytes in 64-bit objects.  Skip notes
>      with incorrect alignment.  */
>   if (align != (__ELF_NATIVE_CLASS / 8))
>     return;
>

This code is correct.  NT_GNU_PROPERTY_TYPE_0 follows gABI.


-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/16/2018 03:39 PM, H.J. Lu wrote:
> On Thu, Aug 16, 2018 at 6:31 AM, Florian Weimer <fweimer@redhat.com> wrote:
>> On 08/16/2018 03:19 PM, H.J. Lu wrote:
>>>
>>> On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com>
>>> wrote:
>>>>
>>>> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>>>>
>>>>>
>>>>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>>>>> added to Linux Extensions to gABI:
>>>>>
>>>>> https://github.com/hjl-tools/linux-abi
>>>>>
>>>>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>>>>> processor-specific program property types for i386 and x86-64.
>>>>
>>>>
>>>>
>>>> The specification is incomplete as far as alignment matters are
>>>> concerned.
>>>
>>>
>>> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
>>>
>>> has
>>>
>>> 2.1.7 Alignment of Note Sections
>>>
>>> All entries in a PT_NOTE segment have the same alignment which equals to
>>> the
>>> p_align field in program header.
>>> According to gABI, each note entry should be aligned to 4 bytes in 32-bit
>>> objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
>>> Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
>>> aligned
>>> to 4 bytes in both 32-bit and 64-bit objects. Note parser should use
>>> p_align for
>>> note alignment, instead of assuming alignment based on ELF file class.
>>
>>
>> This is still ambiguous, particularly based on your comments below.
> 
> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
> 
> conforms to gABI unless stated otherwise.

I still think there is ambiguity.

> I was wrong.  We need 2 NOTE segments one fore 8-byte alignment and
> one for 4-byte alignment.

If one needs to generate two segments, the ABI documents should say so. 
Otherwise, linker implementors will assume that there are other ways to 
implement this.

> glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
> since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
> generates 4 byte alignment, it is a gold bug.

See above.  I don't have a strong opinion how we make gold and glibc 
interoperate, but if gold is wrong, the ABI documents should be made 
more precise.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/16/2018 03:39 PM, H.J. Lu wrote:
> glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
> since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
> generates 4 byte alignment, it is a gold bug.

I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 7:01 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 08/16/2018 03:39 PM, H.J. Lu wrote:
>>
>> On Thu, Aug 16, 2018 at 6:31 AM, Florian Weimer <fweimer@redhat.com>
>> wrote:
>>>
>>> On 08/16/2018 03:19 PM, H.J. Lu wrote:
>>>>
>>>>
>>>> On Thu, Aug 16, 2018 at 6:00 AM, Florian Weimer <fweimer@redhat.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> On 08/07/2018 10:41 PM, H.J. Lu wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> The .note.gnu.property section with NT_GNU_PROPERTY_TYPE_0 has been
>>>>>> added to Linux Extensions to gABI:
>>>>>>
>>>>>> https://github.com/hjl-tools/linux-abi
>>>>>>
>>>>>> GNU_PROPERTY_X86_ISA_1_USED and GNU_PROPERTY_X86_ISA_1_NEEDED are
>>>>>> processor-specific program property types for i386 and x86-64.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> The specification is incomplete as far as alignment matters are
>>>>> concerned.
>>>>
>>>>
>>>>
>>>> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
>>>>
>>>> has
>>>>
>>>> 2.1.7 Alignment of Note Sections
>>>>
>>>> All entries in a PT_NOTE segment have the same alignment which equals to
>>>> the
>>>> p_align field in program header.
>>>> According to gABI, each note entry should be aligned to 4 bytes in
>>>> 32-bit
>>>> objects or 8 bytes in 64-bit objects. But .note.ABI-tag section (see
>>>> Section 2.1.6) and .note.gnu.build-id section (see Section 2.1.4) are
>>>> aligned
>>>> to 4 bytes in both 32-bit and 64-bit objects. Note parser should use
>>>> p_align for
>>>> note alignment, instead of assuming alignment based on ELF file class.
>>>
>>>
>>>
>>> This is still ambiguous, particularly based on your comments below.
>>
>>
>> https://github.com/hjl-tools/linux-abi/wiki/linux-abi-draft.pdf
>>
>> conforms to gABI unless stated otherwise.
>
>
> I still think there is ambiguity.

Can you check out

https://github.com/hjl-tools/linux-abi

and send me a patch to make note alignment less ambiguous?


-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 7:21 AM, Florian Weimer <fweimer@redhat.com> wrote:
> On 08/16/2018 03:39 PM, H.J. Lu wrote:
>>
>> glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
>> since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
>> generates 4 byte alignment, it is a gold bug.
>
>
> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535
>

A patch is posted at:

https://sourceware.org/ml/binutils/2018-08/msg00306.html

-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Mark Wielaard]

On Thu, Aug 16, 2018 at 04:21:05PM +0200, Florian Weimer wrote:
> On 08/16/2018 03:39 PM, H.J. Lu wrote:
> > glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
> > since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
> > generates 4 byte alignment, it is a gold bug.
> 
> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535

I don't think this is a bug in gold, but one in ld:
https://sourceware.org/bugzilla/show_bug.cgi?id=22749

In the GNU abi all ELF Notes are arrays of 32bit words
(and so 4-byte aligned). This is the same for most other
ELF systems. Making the ELF Notes fields 64bit words (and
so 8-byte aligned) in ELFCLASS64 would indeed be what gabi
literally says, but not what GNU systems, and others, follow.

Having a mix of 4-byte words and 8-byte words ELF Notes
in the same ELF file seems unnecessarily confusing and
introduces extra segments and sections.

Cheers,

Mark

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 12:16 PM, Mark Wielaard <mark@klomp.org> wrote:
> On Thu, Aug 16, 2018 at 04:21:05PM +0200, Florian Weimer wrote:
>> On 08/16/2018 03:39 PM, H.J. Lu wrote:
>> > glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
>> > since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
>> > generates 4 byte alignment, it is a gold bug.
>>
>> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535
>
> I don't think this is a bug in gold, but one in ld:
> https://sourceware.org/bugzilla/show_bug.cgi?id=22749
>
> In the GNU abi all ELF Notes are arrays of 32bit words
> (and so 4-byte aligned). This is the same for most other

That is not true.  NT_GNU_PROPERTY_TYPE_0 note is
8 byte aligned in 64-bit ELF.

> ELF systems. Making the ELF Notes fields 64bit words (and
> so 8-byte aligned) in ELFCLASS64 would indeed be what gabi
> literally says, but not what GNU systems, and others, follow.
>
> Having a mix of 4-byte words and 8-byte words ELF Notes
> in the same ELF file seems unnecessarily confusing and
> introduces extra segments and sections.
>
> Cheers,
>
> Mark



-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Mark Wielaard]

On Thu, Aug 16, 2018 at 12:36:34PM -0700, H.J. Lu wrote:
> On Thu, Aug 16, 2018 at 12:16 PM, Mark Wielaard <mark@klomp.org> wrote:
> > On Thu, Aug 16, 2018 at 04:21:05PM +0200, Florian Weimer wrote:
> >> On 08/16/2018 03:39 PM, H.J. Lu wrote:
> >> > glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
> >> > since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
> >> > generates 4 byte alignment, it is a gold bug.
> >>
> >> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535
> >
> > I don't think this is a bug in gold, but one in ld:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=22749
> >
> > In the GNU abi all ELF Notes are arrays of 32bit words
> > (and so 4-byte aligned). This is the same for most other
> 
> That is not true.  NT_GNU_PROPERTY_TYPE_0 note is
> 8 byte aligned in 64-bit ELF.

Which is the bug we are trying to fix, right?
The issue with that note is that ld generates it with 32bit words,
but with extra padding to make it 8-byte aligned. That makes it
not confirm to the GNU abi and also not to the gabi. It causes all
ELF Note parsers to have to be adjusted. And since there are also
normal GNU abi ELF Notes without the extra padding it causes extra
sections and segments to be created because the alignment differs.
It seems simpler to just fix ld so that glibc can use a normal ELF
note parser for all ELF Notes, than to propagate that bug to other
linkers.

> > ELF systems. Making the ELF Notes fields 64bit words (and
> > so 8-byte aligned) in ELFCLASS64 would indeed be what gabi
> > literally says, but not what GNU systems, and others, follow.
> >
> > Having a mix of 4-byte words and 8-byte words ELF Notes
> > in the same ELF file seems unnecessarily confusing and
> > introduces extra segments and sections.
> >
> > Cheers,
> >
> > Mark
> 
> 
> 
> -- 
> H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/16/2018 09:16 PM, Mark Wielaard wrote:
> On Thu, Aug 16, 2018 at 04:21:05PM +0200, Florian Weimer wrote:
>> On 08/16/2018 03:39 PM, H.J. Lu wrote:
>>> glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
>>> since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
>>> generates 4 byte alignment, it is a gold bug.
>>
>> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535
> 
> I don't think this is a bug in gold, but one in ld:
> https://sourceware.org/bugzilla/show_bug.cgi?id=22749
> 
> In the GNU abi all ELF Notes are arrays of 32bit words
> (and so 4-byte aligned). This is the same for most other
> ELF systems. Making the ELF Notes fields 64bit words (and
> so 8-byte aligned) in ELFCLASS64 would indeed be what gabi
> literally says, but not what GNU systems, and others, follow.

The binaries are already out there and will be around for a while, even 
if we reduce the alignment now.  So generic ELF parsers need to cope 
with these binaries anyway.

> Having a mix of 4-byte words and 8-byte words ELF Notes
> in the same ELF file seems unnecessarily confusing and
> introduces extra segments and sections.

There are no 8-byte word ELF notes on GNU systems, that's an HP-UX 
feature.  The new 8-byte-aligned notes still have a four-byte header.

But like you, I don't yet see the value of the 8-byte alignment.  We 
could decide that the current gold behavior is valid, fix glibc, and 
move on.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Mark Wielaard]

On Fri, Aug 17, 2018 at 08:20:53AM +0200, Florian Weimer wrote:
> > Having a mix of 4-byte words and 8-byte words ELF Notes
> > in the same ELF file seems unnecessarily confusing and
> > introduces extra segments and sections.
> 
> There are no 8-byte word ELF notes on GNU systems, that's an HP-UX feature.
> The new 8-byte-aligned notes still have a four-byte header.

Yes, you are right, HP-UX follows gabi and has 64-bit word notes,
with 8-byte alignment. I meant the different padding used with GNU
style 32-bit word ELF notes to get different alignment styles which
is confusing and none compliant to any standard.

> But like you, I don't yet see the value of the 8-byte alignment.  We could
> decide that the current gold behavior is valid, fix glibc, and move on.

Right. gold seems to produce normal GNU abi ELF Notes, which should
be accepted as is.

Cheers,

Mark

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 11:41 PM, Mark Wielaard <mark@klomp.org> wrote:
> On Fri, Aug 17, 2018 at 08:20:53AM +0200, Florian Weimer wrote:
>> > Having a mix of 4-byte words and 8-byte words ELF Notes
>> > in the same ELF file seems unnecessarily confusing and
>> > introduces extra segments and sections.
>>
>> There are no 8-byte word ELF notes on GNU systems, that's an HP-UX feature.
>> The new 8-byte-aligned notes still have a four-byte header.
>
> Yes, you are right, HP-UX follows gabi and has 64-bit word notes,
> with 8-byte alignment. I meant the different padding used with GNU
> style 32-bit word ELF notes to get different alignment styles which
> is confusing and none compliant to any standard.
>
>> But like you, I don't yet see the value of the 8-byte alignment.  We could
>> decide that the current gold behavior is valid, fix glibc, and move on.
>
> Right. gold seems to produce normal GNU abi ELF Notes, which should
> be accepted as is.
>

NT_GNU_PROPERTY_TYPE_0 should stay to follow gABI.


-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Thu, Aug 16, 2018 at 11:20 PM, Florian Weimer <fweimer@redhat.com> wrote:
> On 08/16/2018 09:16 PM, Mark Wielaard wrote:
>>
>> On Thu, Aug 16, 2018 at 04:21:05PM +0200, Florian Weimer wrote:
>>>
>>> On 08/16/2018 03:39 PM, H.J. Lu wrote:
>>>>
>>>> glibc only discards 4-byte aligned NT_GNU_PROPERTY_TYPE_0 note
>>>> since NT_GNU_PROPERTY_TYPE_0 note follows gABI.  If gold
>>>> generates 4 byte alignment, it is a gold bug.
>>>
>>>
>>> I filed: https://sourceware.org/bugzilla/show_bug.cgi?id=23535
>>
>>
>> I don't think this is a bug in gold, but one in ld:
>> https://sourceware.org/bugzilla/show_bug.cgi?id=22749
>>
>> In the GNU abi all ELF Notes are arrays of 32bit words
>> (and so 4-byte aligned). This is the same for most other
>> ELF systems. Making the ELF Notes fields 64bit words (and
>> so 8-byte aligned) in ELFCLASS64 would indeed be what gabi
>> literally says, but not what GNU systems, and others, follow.
>
>
> The binaries are already out there and will be around for a while, even if
> we reduce the alignment now.  So generic ELF parsers need to cope with these
> binaries anyway.
>
>> Having a mix of 4-byte words and 8-byte words ELF Notes
>> in the same ELF file seems unnecessarily confusing and
>> introduces extra segments and sections.
>
>
> There are no 8-byte word ELF notes on GNU systems, that's an HP-UX feature.
> The new 8-byte-aligned notes still have a four-byte header.
>
> But like you, I don't yet see the value of the 8-byte alignment.  We could
> decide that the current gold behavior is valid, fix glibc, and move on.

Gold didn't follow the spec and should be fixed.

-- 
H.J.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Mark Wielaard]

On Fri, Aug 17, 2018 at 08:10:15AM -0700, H.J. Lu wrote:
> On Thu, Aug 16, 2018 at 11:41 PM, Mark Wielaard <mark@klomp.org> wrote:
> > On Fri, Aug 17, 2018 at 08:20:53AM +0200, Florian Weimer wrote:
> >> But like you, I don't yet see the value of the 8-byte alignment.  We could
> >> decide that the current gold behavior is valid, fix glibc, and move on.
> >
> > Right. gold seems to produce normal GNU abi ELF Notes, which should
> > be accepted as is.
> 
> NT_GNU_PROPERTY_TYPE_0 should stay to follow gABI.

That doesn't make any sense. gABI doesn't have a concept of 32bit word
ELF notes that are 8-byte aligned. That is just a bug in ld when it
generates the note. It also doesn't make sense to generate GNU ELF notes
with slightly different padding added depending on type in the same ELF
file. That just creates confusion, causes you to define different
alignments of notes resulting in extra PT_LOAD segments and results in
bugs like we are discussing now where ELF note parsers fail to parse
some (valid) notes. Lets just agree that the gold linker is correct and
produces consistent GNU abi ELF notes. And lets just fix ld to do the same.

Thanks,

Mark

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Cary Coutant]

>> >> But like you, I don't yet see the value of the 8-byte alignment.  We could
>> >> decide that the current gold behavior is valid, fix glibc, and move on.
>> >
>> > Right. gold seems to produce normal GNU abi ELF Notes, which should
>> > be accepted as is.
>>
>> NT_GNU_PROPERTY_TYPE_0 should stay to follow gABI.
>
> That doesn't make any sense. gABI doesn't have a concept of 32bit word
> ELF notes that are 8-byte aligned. That is just a bug in ld when it
> generates the note. It also doesn't make sense to generate GNU ELF notes
> with slightly different padding added depending on type in the same ELF
> file. That just creates confusion, causes you to define different
> alignments of notes resulting in extra PT_LOAD segments and results in
> bugs like we are discussing now where ELF note parsers fail to parse
> some (valid) notes. Lets just agree that the gold linker is correct and
> produces consistent GNU abi ELF notes. And lets just fix ld to do the same.

I thought the outcome of the old discussion on the gABI/psABI mailing
lists was that there was no value in breaking compatibility by
introducing an 8-byte aligned note section.

As it is, the psABI definition of the gnu properties note is a
definitional disaster: like a proper note, it's got three 4-byte words
(namesz, descsz, and type), followed by the name field (which happens
to be 4 bytes long in this case), then a desc field. For this note,
the desc field contains a properties array, where each property has
two 4-byte values (pr_type and pr_datasz) and a data field padded to a
multiple of 8 bytes. But it describes the properties array as an array
of 8-byte words, even though it's really composed of the two 4-byte
words and an arbitrary-length buffer (which according to the spec, is
always 8 bytes). Why an 8-byte alignment, though? It doesn't provide
any conceivable benefit.

I think gold is doing the right thing by placing the output in a
4-byte aligned note section, and combining it with other note sections
in a 4-byte aligned PT_NOTE segment.

As long as HJ is planning on breaking compatibility with his new
definition of the x86 properties [1], why not just throw it away and
do it right? Let's stop using NOTE sections for something they weren't
designed for, and instead use a special section type and segment type
(in keeping with existing models like Sun, HP, MIPS, and Arm). Why
should the loader have to go parsing everything in a generic PT_NOTE
segment anyway?

-cary

[1] https://groups.google.com/d/msg/x86-64-abi/-D05GQ3kWrA/stKtIPy8DQAJ

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/22/2018 01:02 AM, Cary Coutant wrote:
>>>>> But like you, I don't yet see the value of the 8-byte alignment.  We could
>>>>> decide that the current gold behavior is valid, fix glibc, and move on.
>>>>
>>>> Right. gold seems to produce normal GNU abi ELF Notes, which should
>>>> be accepted as is.
>>>
>>> NT_GNU_PROPERTY_TYPE_0 should stay to follow gABI.
>>
>> That doesn't make any sense. gABI doesn't have a concept of 32bit word
>> ELF notes that are 8-byte aligned. That is just a bug in ld when it
>> generates the note. It also doesn't make sense to generate GNU ELF notes
>> with slightly different padding added depending on type in the same ELF
>> file. That just creates confusion, causes you to define different
>> alignments of notes resulting in extra PT_LOAD segments and results in
>> bugs like we are discussing now where ELF note parsers fail to parse
>> some (valid) notes. Lets just agree that the gold linker is correct and
>> produces consistent GNU abi ELF notes. And lets just fix ld to do the same.
> 
> I thought the outcome of the old discussion on the gABI/psABI mailing
> lists was that there was no value in breaking compatibility by
> introducing an 8-byte aligned note section.

The code for generating them is still in BFD ld and has been there for 
several binutils releases.  <cet.h> in GCC 8 generates 8-byte-aligned 
notes, too.

glibc 2.28 assumes that no 4-byte-aligned PT_NOTE segments contain GNU 
property notes and will search for such notes in 8-byte aligned segments 
only.  The glibc changes were formally reviewed and checked for 
interoperability with BFD ld.  I think we have (had) community consensus 
for them.

We have built and hundreds (if not thousands) of binaries with 
8-byte-aligned notes and shipped them (in Fedora), assuming that there 
was upstream consensus about this x86-64 psABI extension.

> As it is, the psABI definition of the gnu properties note is a
> definitional disaster: like a proper note, it's got three 4-byte words
> (namesz, descsz, and type), followed by the name field (which happens
> to be 4 bytes long in this case), then a desc field. For this note,
> the desc field contains a properties array, where each property has
> two 4-byte values (pr_type and pr_datasz) and a data field padded to a
> multiple of 8 bytes. But it describes the properties array as an array
> of 8-byte words, even though it's really composed of the two 4-byte
> words and an arbitrary-length buffer (which according to the spec, is
> always 8 bytes). Why an 8-byte alignment, though? It doesn't provide
> any conceivable benefit.
> 
> I think gold is doing the right thing by placing the output in a
> 4-byte aligned note section, and combining it with other note sections
> in a 4-byte aligned PT_NOTE segment.

Based on what I've seen, this is a technically feasible option.

But we have existing binaries with 8-byte-aligned notes, and those won't 
go away soon.  There is a cost to having 8-byte-aligned notes (gold 
fixes, elfutils fixes, plenty of other fixes), and to having 
4-byte-aligned notes only (CET-compatible binaries will not be treated 
as such because their 8-byte-aligned PT_NOTE section is invalid).

I don't know if there is anything else that is coming down the pipe 
which would benefit from 8-byte-aligned notes.  Nick has been working on 
.gnu.build.attributes non-allocated note sections.  They are nominally 
4-byte-aligned, but contain 64-bit relocations, but so far, the linker 
has not complained, so maybe this is okay.

> As long as HJ is planning on breaking compatibility with his new
> definition of the x86 properties [1], why not just throw it away and
> do it right? Let's stop using NOTE sections for something they weren't
> designed for, and instead use a special section type and segment type
> (in keeping with existing models like Sun, HP, MIPS, and Arm).

This, too, would invalidate existing binaries.

> Why should the loader have to go parsing everything in a generic PT_NOTE
> segment anyway?

With 8-byte alignment, in practice, it is currently quite fast to 
discover the new notes because the other notes have 4-byte alignment, so 
the PT_NOTE segment with the property notes should be pleasantly short.

Again, maybe we shouldn't have built all those binaries, but we thought 
the binutils community at least was in favor of the 8-byte-aligned 
notes,  This puts me in a difficult decision: The technical facts seem 
to favor 4-byte-aligned notes (but again, there might be further 
features that actually need 8-byte alignment).  But the existing 
binaries with 8-byte-aligned notes provide a strong business 
justification for Red Hat to keep support for the 8-byte-aligned notes.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Mark Wielaard]

On Wed, Aug 22, 2018 at 11:39:02AM +0200, Florian Weimer wrote:
> glibc 2.28 assumes that no 4-byte-aligned PT_NOTE segments contain GNU
> property notes and will search for such notes in 8-byte aligned segments
> only.  The glibc changes were formally reviewed and checked for
> interoperability with BFD ld.  I think we have (had) community consensus for
> them.

I think there was actually consensus that they were a bad idea.
There is a bug and a fix for BFD ld:
https://sourceware.org/bugzilla/show_bug.cgi?id=22749

Unfortunately it didn't make it before the binutils release:
https://sourceware.org/ml/binutils/2018-01/msg00354.html
Which might have given the impression it wasn't a simple bug.
Sorry about that. I have raised the issue a couple of time,
but apparently not clearly enough.

I think the best thing forward is to actually apply and backport
the fix now (and fix cet.h) to generate proper GNU ELF notes.
It is unfortunate there are a few binaries out there with
these bad notes. But they haven't hit any long term distros
yet. And they should impact only CET, which isnt really
supported anywhere yet because there isn't any hardware
for it.

I understand the appeal of introducing a new section and
segment type for storing these properties, but I fear that
will be much more effort and coordination.

Cheers,

Mark

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Cary Coutant]

>> Why should the loader have to go parsing everything in a generic PT_NOTE
>> segment anyway?
>
> With 8-byte alignment, in practice, it is currently quite fast to discover
> the new notes because the other notes have 4-byte alignment, so the PT_NOTE
> segment with the property notes should be pleasantly short.

I understand the benefit of having a PT_NOTE segment containing
nothing but the one note, and this is certainly expedient, but it's
quite ugly and, I think, fragile. This really underscores the value of
defining a bespoke segment type for this one purpose.

How difficult would it be for glibc if the linker would consume the
existing .note.gnu.property sections, then generate an output section
with the same format (maybe minus the SHT_NOTE overhead), but using a
new section type in its own new segment type? You could continue to
look for 8-byte aligned legacy PT_NOTE segments, but could also look
for the new PT_GNU_PROPERTY segment.

-cary

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Michael Matz]

Hi,

On Wed, 22 Aug 2018, Florian Weimer wrote:

> glibc 2.28 assumes that no 4-byte-aligned PT_NOTE segments contain GNU 
> property notes and will search for such notes in 8-byte aligned segments 
> only. The glibc changes were formally reviewed and checked for 
> interoperability with BFD ld.  I think we have (had) community consensus 
> for them.

Not really.  You had one person favoring 8byte alignment (variously 
because the gABI says so, even though that's irrelevant for .gnu.note, 
and/or because then int64 accesses can be used without having to write a 
read_int64 macro that deals with unalignment), and the rest either 
ignoring this or saying "ugh, bad idea, breaks compatibility and breaks 
forward compat because not self-describing anymore".  The former put in 
work to implement 8byte alignment, ignoring the compat issues.  Now you 
actually see those issues.  Everybody looses.  That's what you get if 
people create facts before consensus.


Ciao,
Michael.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/23/2018 04:41 PM, Michael Matz wrote:
> Hi,
> 
> On Wed, 22 Aug 2018, Florian Weimer wrote:
> 
>> glibc 2.28 assumes that no 4-byte-aligned PT_NOTE segments contain GNU
>> property notes and will search for such notes in 8-byte aligned segments
>> only. The glibc changes were formally reviewed and checked for
>> interoperability with BFD ld.  I think we have (had) community consensus
>> for them.
> 
> Not really.

Sorry for being ambiguous.  I was talking about review of the glibc 
changes, not the binutils changes in the paragraph above.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

On 08/23/2018 01:36 AM, Cary Coutant wrote:
>>> Why should the loader have to go parsing everything in a generic PT_NOTE
>>> segment anyway?
>>
>> With 8-byte alignment, in practice, it is currently quite fast to discover
>> the new notes because the other notes have 4-byte alignment, so the PT_NOTE
>> segment with the property notes should be pleasantly short.
> 
> I understand the benefit of having a PT_NOTE segment containing
> nothing but the one note, and this is certainly expedient, but it's
> quite ugly and, I think, fragile. This really underscores the value of
> defining a bespoke segment type for this one purpose.

Yes, the existing note processing in linkers makes this fragile.

> How difficult would it be for glibc if the linker would consume the
> existing .note.gnu.property sections, then generate an output section
> with the same format (maybe minus the SHT_NOTE overhead), but using a
> new section type in its own new segment type? You could continue to
> look for 8-byte aligned legacy PT_NOTE segments, but could also look
> for the new PT_GNU_PROPERTY segment.

It does not sound difficult to implement.  This is based on the 
assumption that the current dynamic loader ignores unknown segment types 
(which I think is true).

But I don't view this a win because it doesn't take the 8-byte notes out 
of circulation.  We'd still have to support both indefinitely, and would 
have update GCC as well.  We can just keep using the existing code.

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

At the GNU Tools Cauldron in Manchester, we had an ad-hoc side meeting
to work out a consensus among many of those involved so far.

I'm including the minutes below.  It is my hope that this can serve as a
guide for obtaining community consensus.

Cary, after reviewing the minutes, would you be willing to join the
emerging consensus, and re-review H.J.'s patch to align gold's behavior
with BFD ld?

Thanks,
Florian

# Background for the GNU property notes design

The notes are intended for subarchitecture markup, including the recent
Intel CET (CET).  (CET is an upcoming Intel CPU feature which needs
forwards and backwards compatibility for binaries, across and with the
existing ecosystem.  CPU features need to be enabled per process if all
loaded objects support it.)

Putting the markup in a new, separate program header segment (PT_*)
has more risk across the ecosystem than PT_NOTE.  We know that PT_NOTE
works because different new notes have been added to the PT_NOTE
segment over time.

gABI does not completely and unambiguously describe the layout of note
segments and sections and the format of the note header, although the
historic interpretation has been that the note header has three 32-bit
words, independently of the ELF class, particularly on GNU/Linux.
(But HP-UX uses 8-byte words in the note header in the 64-bit case.)

gABI does say that ELF notes should be 8-byte aligned for the 64-bit
ELF class.  But there are commonly used existing 4-byte-aligned notes:
the ABI tag and the build ID.  The Linux kernel, when generating
coredumps, may put 0 or 1 in the p_align field of the PT_NOTE segment;
this is a clear bug, the notes are actually 4-byte-aligned.

There could be relocations in the note; nothing currently forbids
that. The stack size in GNU_PROPERTY_STACK_SIZE is 64-bit for the 64-bit
ELF class (> 4 GiB may not make sense, but is theoretically possible).
x86-64 does not need alignment, but strict alignment targets need it,
including perhaps for relocations.

Therefore, a conscious decision was made to follow gABI and use 8-byte
alignment for the new note types (GNU property notes).

# Current state

8-byte aligned notes have been in binutils (BFD ld) for two years.
GCC 8 includes them (in <cet.h>).  glibc 2.28 ignores property notes
which are not 8-byte-aligned.

Distributions have built binaries with 8-byte-aligned GNU property notes.

Current binutils gold merges all allocated note section into a 4-byte
aligned note segment.

Relocations in notes work, and annobin uses 8-byte relocations in
4-byte-aligned notes in relocatable (ET_REL) objects on multiple
64-bit targets.

Due to the kernel bug which generates PT_NOTE headers with p_align
having values 0 or 1, some consumers ignore the p_align field and
always assume 4.

# Some available options

(1) Only 4-byte-aligned notes are valid.  Existing binaries with 8-byte
alignment become invalid.

(2) The current 8+4 alignment mix (8-byte alignment for GNU Property
notes on ELFCLASS64, 4-byte alignment for older note types such as ABI
tag and build ID) as implemented by BFD ld, GCC, and glibc.

(3) Change GCC and linkers to generate 4-byte-aligned notes
(producers), but keep backwards compatibility with existing
8-byte-aligned notes in binaries (in consumers).

(4) A new PT_* segment for property notes.

# Consensus position of those present

(4) Does not have working code today, so it was not considered
further. (It is also considered larger risk, as discussed above.)

A simplification over the status quo is not achievable with (3)
because consumers still need to be updated.  This means the choice is
between (1) and (2).

(1) makes existing binaries invalid, and there was general agreement
that this is a bad idea.  It also fails to support notes with relocation
on ELFCLASS64 strict-alignment targets.

This leaves us with (2).  There was agreement that link editor
behavior for this approach is currently underspecified.  Some
ABI-level document should indicate what linkers should do (for
example, produce separate PT_NOTE segments for different alignments,
report link failures for note sections of different alignment and the
same name in relocatable links).

In order to obtain consensus among those present, we agreed that option
(2) (that is, 8+4 alignment mix as implemented in BFD ld, GCC, and glibc
today) should only be adopted with 8-byte-alignment for the GNU property
notes, and not automatically for all future notes.  No precedent for
future notes should be established by this decision.

# Further actions

If the community adopts the consensus of those present at the meeting,
binutils gold should change to adapt to BFD ld, but only for
.note.gnu.property.

Florian agreed to try to document the expected link editor behavior
regarding notes and note merging.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Michael Matz]

Hi,

(didn't take part in the side meeting, so sorry if this was discussed)

On Wed, 19 Sep 2018, Florian Weimer wrote:

> Distributions have built binaries with 8-byte-aligned GNU property 
> notes.

Which ones?  I know that we (SUSE) avoided CET-marked binaries for the 
reason we are in the current predicament.  I.e. is that _really_ a problem 
at all (or a large one)?

Thing is, there are also binaries (or there could be) that are currently 
"valid" that the align-8 approach makes invalid, so why say that's a 
problem for the align-4 approach, but not for the align-8 one?

> (1) makes existing binaries invalid, and there was general agreement
> that this is a bad idea.  It also fails to support notes with relocation
> on ELFCLASS64 strict-alignment targets.


Ciao,
Michael.

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Florian Weimer]

* Michael Matz:

> (didn't take part in the side meeting, so sorry if this was discussed)

I don't think this came up.

> On Wed, 19 Sep 2018, Florian Weimer wrote:
>
>> Distributions have built binaries with 8-byte-aligned GNU property 
>> notes.
>
> Which ones?

Fedora 29 and downstream.

> Thing is, there are also binaries (or there could be) that are currently 
> "valid" that the align-8 approach makes invalid, so why say that's a 
> problem for the align-4 approach, but not for the align-8 one?

The 4-byte-aligned GNU property notes are ignored by glibc even if they
are otherwise syntactically valid.  So even if there is a hypothetical
distribution that is mostly built with binutils gold and thus has
4-byte-aligned GNU property notes, it would still not work as intended,
which makes me believe that such a distribution does not exist. 8-)

Thanks,
Florian

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[Cary Coutant]

> At the GNU Tools Cauldron in Manchester, we had an ad-hoc side meeting
> to work out a consensus among many of those involved so far.
>
> I'm including the minutes below.  It is my hope that this can serve as a
> guide for obtaining community consensus.
>
> Cary, after reviewing the minutes, would you be willing to join the
> emerging consensus, and re-review H.J.'s patch to align gold's behavior
> with BFD ld?

Yes, I will yield to the consensus, and I'll change gold's treatment
of the new note section.

But please allow me to express my disappointment and a few minor
quibbles for the record....

> Putting the markup in a new, separate program header segment (PT_*)
> has more risk across the ecosystem than PT_NOTE.  We know that PT_NOTE
> works because different new notes have been added to the PT_NOTE
> segment over time.

It's discouraging that the community is so reluctant to make use of
the established and well-proven extension mechanism that was designed
into ELF. An extension mechanism that people are afraid to use is not
an extension mechanism. I think it's a bit hyperbolic to imply that we
*don't* know that a new segment type for this purpose will work, since
that very mechanism has already been put to use on at least 3
platforms.

> gABI does not completely and unambiguously describe the layout of note
> segments and sections and the format of the note header, although the
> historic interpretation has been that the note header has three 32-bit
> words, independently of the ELF class, particularly on GNU/Linux.
> (But HP-UX uses 8-byte words in the note header in the 64-bit case.)

I think it's more fair to say that the gABI does in fact completely
and unambiguously describe the layout of 64-bit note sections, but
that Sun and Gnu accidentally misread or ignored the spec in their
implementations, establishing a precedent that led to confusion and
compatibility issues. The choice to use 8-byte alignment for this note
-- ignoring public objections to the design -- added even more
confusion.

> # Some available options
>
> (1) Only 4-byte-aligned notes are valid.  Existing binaries with 8-byte
> alignment become invalid.
>
> (2) The current 8+4 alignment mix (8-byte alignment for GNU Property
> notes on ELFCLASS64, 4-byte alignment for older note types such as ABI
> tag and build ID) as implemented by BFD ld, GCC, and glibc.
>
> (3) Change GCC and linkers to generate 4-byte-aligned notes
> (producers), but keep backwards compatibility with existing
> 8-byte-aligned notes in binaries (in consumers).
>
> (4) A new PT_* segment for property notes.
>
> # Consensus position of those present
>
> (4) Does not have working code today, so it was not considered
> further. (It is also considered larger risk, as discussed above.)
>
> A simplification over the status quo is not achievable with (3)
> because consumers still need to be updated.  This means the choice is
> between (1) and (2).

If the solution must meet the criterion that consumers don't need to
be updated, then (1) is not a choice, as you noted below. If that was
indeed an immutable criterion, this meeting wasn't really necessary,
was it?

> (1) makes existing binaries invalid, and there was general agreement
> that this is a bad idea.  It also fails to support notes with relocation
> on ELFCLASS64 strict-alignment targets.
>
> This leaves us with (2).  There was agreement that link editor
> behavior for this approach is currently underspecified.  Some
> ABI-level document should indicate what linkers should do (for
> example, produce separate PT_NOTE segments for different alignments,
> report link failures for note sections of different alignment and the
> same name in relocatable links).
>
> In order to obtain consensus among those present, we agreed that option
> (2) (that is, 8+4 alignment mix as implemented in BFD ld, GCC, and glibc
> today) should only be adopted with 8-byte-alignment for the GNU property
> notes, and not automatically for all future notes.  No precedent for
> future notes should be established by this decision.

I'm happy you've decided to restrict this solution to just the one
note. I'd prefer, however, that we do establish a precedent that this
should be the only exception, and that *all* future notes must follow
the a priori convention of using 4-byte aligned notes (on those
platforms that did not implement according to the gABI spec).

> Florian agreed to try to document the expected link editor behavior
> regarding notes and note merging.

Some requirements I'd like to see:

(1) An NT_GNU_PROPERTY_TYPE_0 note section must be named
".note.gnu.property" with type SHT_NOTE.

(2) A ".note.gnu.property" section must contain one and only one note,
whose type must be NT_GNU_PROPERTY_TYPE_0.

(3) A ".note.gnu.property" section must have sh_align of 8, and its
length must be a multiple of 8.

(4) A linker is expected to combine ".note.gnu.property" sections in a
manner described by the ABI documentation, and place the combined
result, as a single note, in a unique SHT_NOTE section named
".note.gnu.property", and in a unique PT_NOTE segment.

-cary

Re: PT_NOTE alignment, NT_GNU_PROPERTY_TYPE_0, glibc and gold

[H.J. Lu]

On Wed, Sep 26, 2018 at 10:39 AM, Cary Coutant <ccoutant@gmail.com> wrote:
>
> Some requirements I'd like to see:

NT_GNU_PROPERTY_TYPE_0 is documented at

https://github.com/hjl-tools/linux-abi

> (1) An NT_GNU_PROPERTY_TYPE_0 note section must be named
> ".note.gnu.property" with type SHT_NOTE.

If it isn't specified this way, please send a patch.

> (2) A ".note.gnu.property" section must contain one and only one note,
> whose type must be NT_GNU_PROPERTY_TYPE_0.

True for linker output and all properties must be sorted.   For relocatable
inputs, there may be more than one NT_GNU_PROPERTY_TYPE_0 notes:

[hjl@gnu-cfl-1 tmp]$ touch n.c
[hjl@gnu-cfl-1 tmp]$ gcc -fcf-protection -c n.c -Wa,-mx86-used-note=yes
[hjl@gnu-cfl-1 tmp]$ readelf -n n.o

Displaying notes found in: .note.gnu.property
  Owner                 Data size Description
  GNU                  0x00000010 NT_GNU_PROPERTY_TYPE_0
      Properties: x86 feature: IBT, SHSTK
  GNU                  0x00000020 NT_GNU_PROPERTY_TYPE_0
      Properties: x86 ISA used: <None>
x86 feature used: x86
[hjl@gnu-cfl-1 tmp]$

> (3) A ".note.gnu.property" section must have sh_align of 8, and its
> length must be a multiple of 8.

If it isn't specified this way, please send a patch.

> (4) A linker is expected to combine ".note.gnu.property" sections in a
> manner described by the ABI documentation, and place the combined
> result, as a single note, in a unique SHT_NOTE section named
> ".note.gnu.property", and in a unique PT_NOTE segment.
>

Why is a unique PT_NOTE segment required? A PT_NOTE segment
may have more than one SHT_NOTE sections with the same section
alignment and each SHT_NOTE section is padded to section alignment.

-- 
H.J.