public inbox for binutils@sourceware.org
 help / color / mirror / Atom feed
* GNU property saga
@ 2019-03-04 17:25 Michael Matz
  2019-03-04 17:40 ` OR_AND semantics (was: GNU property saga) Michael Matz
  2019-03-20 14:51 ` GNU property saga Mark Wielaard
  0 siblings, 2 replies; 11+ messages in thread
From: Michael Matz @ 2019-03-04 17:25 UTC (permalink / raw)
  To: x86-64-abi; +Cc: binutils

Hello,

I'll admit that I have lost track of the consensus about GNU properties 
and all the discussions last year.  But we want to somewhen get this into 
the psABI document.  I believe there were two points (with subitems) of 
contentions:

1) use PT_NOTE vs PT_xxx for program header containing the properties

I believe the consensus was to use a new PT_xxx value

1a) use new SHT_xxx for sections contain such properties

I believe the consensus was to stay with SHT_NOTE, even though that is
squarely against ELF spirit.

2) semantics of individual ranges of the GNU_PROPERTY_xxx value.

Basically the properties are split into three ranges, where each 
individual member represents itself a set of 32 members (via an uint32) 
representing a feature/property each.  The semantics of the higher level 
(the three ranges) are a fixed set of logical combinations of input 
values:
  * there is a range representing the AND of all input values
  * there is a range representing the OR of all input values
  * there is a range representing something strange (OR_AND), which tries
    to capture the notion of difference between "input-unknown" and 
    "input-known-not-there"

I believe there is consensus on the semantics of the AND and OR range.
Was there ever consensus on the OR_AND thingy?  It strikes me as not 
really implementing what is wanted, especially in relation to future 
extensibility (I'll write a mail about this).

Were there other points of contention, or was that all?

(FWIW, the current textual diff to the psABI document adding GNU 
properties is at 
https://gitlab.com/x86-psABIs/x86-64-ABI/merge_requests/1/diffs )


Ciao,
Michael.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* OR_AND semantics (was: GNU property saga)
  2019-03-04 17:25 GNU property saga Michael Matz
@ 2019-03-04 17:40 ` Michael Matz
  2019-03-05  0:48   ` Cary Coutant
  2019-03-08  3:44   ` H.J. Lu
  2019-03-20 14:51 ` GNU property saga Mark Wielaard
  1 sibling, 2 replies; 11+ messages in thread
From: Michael Matz @ 2019-03-04 17:40 UTC (permalink / raw)
  To: x86-64-abi; +Cc: binutils

[-- Attachment #1: Type: text/plain, Size: 2496 bytes --]

Hi,

On Mon, 4 Mar 2019, Michael Matz wrote:

> Was there ever consensus on the OR_AND thingy?  It strikes me as not 
> really implementing what is wanted, especially in relation to future 
> extensibility (I'll write a mail about this).

So, starting a subthread about only this.  The current description of the 
OR_AND range is this:

----------------
\item[GNU_PROPERTY_X86_UINT32_OR_AND_LO..GNU_PROPERTY_X86_UINT32_OR_AND_HI]
   A bit in the output \code{pr_data} field is set if it is set in any
   relocatable input \code{pr_data} fields and this property is present
   in all relocatable input files.  A missing property implies that its
   bits have unknown values.  When all bits in the the output
   \code{pr_data} field are zero, this property should not be removed
   from output to indicate it has zero in all bits.  If the property
   is in output, all input relocatables have the property.  If the bit
   is 1, some input relocatables have the feature.  If the bit is 0,
	
   none of input relocatables have the feature.
----------------

So, it's clear that the individual bits of the uint32 are independend of 
each other and hence can represent separate features/properties.  This 
means that the phrase "and this property is present in all relocatable 
input files" in not well defined: example situation:

* producer A knows about bits 4 and 5 of FEATURE_X (assumed to be in the 
  OR_AND range), it will set the FEATURE_X property in a.o (and bit 4
  and 5 therein to whatever is correct/requested)
* producer B is older and happens to know only bit 4 of FEATURE_X, so it 
  will set FEATURE_X in b.o, and bit 4 to whatever value is correct, but
  bit 5 will be zero.

Now the above language would include FEATURE_X in the output, but bit 4 
would be reliable (set a.bit4 && b.bit4) while bit 5 would be unreliable 
(it is zero, even if it is one in a.o and _would_ be one in b.o had we 
used a newer producer).

So there's still the confusion between absense and zero-is-unknown and 
zero-is-known, which the OR_AND range was designed to make clear (at least 
as far as I understood it from various mails trying to explain situations 
in which it would help).

So, either I misunderstood the reason of existence for OR_AND, or there's 
some unspelled assumption in the usage of OR_AND (e.g. that there must 
never be producers knowing a different set of bits per FEATURE_xxx uint32 
set), or the language above doesn't capture the intent.

What is it?


Ciao,
Michael.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-04 17:40 ` OR_AND semantics (was: GNU property saga) Michael Matz
@ 2019-03-05  0:48   ` Cary Coutant
  2019-03-06 13:57     ` Michael Matz
  2019-03-08  3:44   ` H.J. Lu
  1 sibling, 1 reply; 11+ messages in thread
From: Cary Coutant @ 2019-03-05  0:48 UTC (permalink / raw)
  To: Michael Matz; +Cc: x86-64-abi, Binutils, Jim Dehnert

> > Was there ever consensus on the OR_AND thingy?  It strikes me as not
> > really implementing what is wanted, especially in relation to future
> > extensibility (I'll write a mail about this).
> ...
> So, it's clear that the individual bits of the uint32 are independend of
> each other and hence can represent separate features/properties.  This
> means that the phrase "and this property is present in all relocatable
> input files" in not well defined: example situation:
>
> * producer A knows about bits 4 and 5 of FEATURE_X (assumed to be in the
>   OR_AND range), it will set the FEATURE_X property in a.o (and bit 4
>   and 5 therein to whatever is correct/requested)
> * producer B is older and happens to know only bit 4 of FEATURE_X, so it
>   will set FEATURE_X in b.o, and bit 4 to whatever value is correct, but
>   bit 5 will be zero.
>
> Now the above language would include FEATURE_X in the output, but bit 4
> would be reliable (set a.bit4 && b.bit4) while bit 5 would be unreliable
> (it is zero, even if it is one in a.o and _would_ be one in b.o had we
> used a newer producer).

I'm guessing that HJ is operating under the premise that if the older
producer B was unaware of bit 5 of FEATURE_X, then the object must
not, in fact, use that feature. His design is all-or-nothing: If the
FEATURE_X property is there, then we have accurate yes/no information
for each feature described by that property. That would work if he's
careful to move a new feature to a completely new FEATURE_Y property
whenever it's possible that an older consumer might have used that
feature before the means of recording it was defined.

Jim and I each suggested an improvement where each feature is
described by two bits: one bit from FEATURE_X_KNOWN and a
corresponding bit from FEATURE_X_USED*. The bits in FEATURE_X_KNOWN
would be ANDed, while the bits in FEATURE_X_USED would be ORed. This
would give us the following four cases:

(a) KNOWN = 0, USED = 0: The feature is not known to be used, but not
all objects provided a value.

(b) KNOWN = 0, USED = 1: The feature is used by at least one object,
but not all objects provided a value.

(c) KNOWN = 1, USED = 0: The feature is not used in any object, and
all objects provided a value.

(d) KNOWN = 1, USED = 1: The feature is used by at least one object,
and all objects provided a value.

Using the OR_AND scheme, case (a) is represented by a missing
property, and case (b) is folded into (a): If not all objects provide
the property, we discard the property, and all features revert to case
(a).

The OR_AND scheme would be equivalent to an OR scheme where missing
properties are treated as all ones -- in that case, we'd end up with
all ones rather than a missing property. For each feature, 0 means we
know the feature is not used, while 1 means we aren't necessarily
sure.

Alternatively, it would be equivalent to an AND scheme where the
meaning of each feature is inverted and missing properties are treated
as all zeroes: 1 means the feature is NOT used, and any object with a
missing property would result in all zeroes. For each feature, 1 means
we know the feature is NOT used, while 0 means we aren't necessarily
sure.

I was trying to understand why HJ didn't care about case (b) -- if a
feature is used by at least one object, what difference does it make
whether we have some objects that didn't say whether they used the
feature? I believe his answer is that the use case for these
properties is to allow the OS to decide whether it can allocate a
program to a downgraded processor, based on whether we know for sure
that the program does *not* use a particular feature. Thus, case (c)
is the important one where the downgrade is feasible, and all other
cases mean that it is not (either because we know the feature is used,
or because we do not know it's not used).

But it seems to me that there's an equally useful purpose for these
properties, if we can distinguish between cases (a) and (b). If we
want to allow the OS to refuse to execute a program that it knows
requires unsupported features, case (b) tells us that we do know the
program uses the feature, and the OS could provide a graceful error
message in that case, as it could for case (d). In case (a), we still
wouldn't know for sure whether the program will run successfully, but
we're no worse off than today.

-cary


* I originally wrote in terms of FEATURE_X_NEEDED, based on a
misstatement in an early email from HJ that led me to believe the
psABI had USED and NEEDED reversed. This scheme works for both USED
and NEEDED, and it's even feasible to use one set of KNOWN bits that
pairs with both a set of USED bits and a set of NEEDED bits.


>
> So there's still the confusion between absense and zero-is-unknown and
> zero-is-known, which the OR_AND range was designed to make clear (at least
> as far as I understood it from various mails trying to explain situations
> in which it would help).
>
> So, either I misunderstood the reason of existence for OR_AND, or there's
> some unspelled assumption in the usage of OR_AND (e.g. that there must
> never be producers knowing a different set of bits per FEATURE_xxx uint32
> set), or the language above doesn't capture the intent.
>
> What is it?
>
>
> Ciao,
> Michael.
>
> --
> You received this message because you are subscribed to the Google Groups "X86-64 System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to x86-64-abi+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-05  0:48   ` Cary Coutant
@ 2019-03-06 13:57     ` Michael Matz
  2019-03-07  6:56       ` Cary Coutant
  0 siblings, 1 reply; 11+ messages in thread
From: Michael Matz @ 2019-03-06 13:57 UTC (permalink / raw)
  To: Cary Coutant; +Cc: x86-64-abi, Binutils, Jim Dehnert

Hello,

On Mon, 4 Mar 2019, Cary Coutant wrote:

> I'm guessing that HJ is operating under the premise that if the older 
> producer B was unaware of bit 5 of FEATURE_X, then the object must not, 
> in fact, use that feature. His design is all-or-nothing: If the 
> FEATURE_X property is there, then we have accurate yes/no information 
> for each feature described by that property. That would work if he's 
> careful to move a new feature to a completely new FEATURE_Y property 
> whenever it's possible that an older consumer might have used that 
> feature before the means of recording it was defined.

Yes, that's the implicit assumption theory I had.

> Jim and I each suggested an improvement where each feature is described 
> by two bits: one bit from FEATURE_X_KNOWN and a corresponding bit from 
> FEATURE_X_USED*. The bits in FEATURE_X_KNOWN would be ANDed, while the 
> bits in FEATURE_X_USED would be ORed. This would give us the following 
> four cases:

And yes, that would be my cut at designing such property feature as well.  
One set of bits that spells out what can be relied upon, and another set 
of bits that contains the actual used/unused state.  Nice, easy, obvious.

> I was trying to understand why HJ didn't care about case (b) -- if a
> feature is used by at least one object, what difference does it make
> whether we have some objects that didn't say whether they used the
> feature? I believe his answer is that the use case for these
> properties is to allow the OS to decide whether it can allocate a
> program to a downgraded processor, based on whether we know for sure
> that the program does *not* use a particular feature. Thus, case (c)
> is the important one where the downgrade is feasible, and all other
> cases mean that it is not (either because we know the feature is used,
> or because we do not know it's not used).

That might be, but truth be told, I'm more comfortable with an orthogonal 
design that can express more than we need right now, than with a design 
where two concepts (known and used) are merged in a non-natural way.

HJ: what's your issue with the KNOWN+USED approach?  I would envision an 
encoding where e.g. even IDs encode the 32 KNOWN bits, and odd IDs encode 
the 32 USED bits.  Optionally we could retain the differentiation between 
OR vs. AND for the logical combination of the USED bits (where the KNOWN 
bits are always ANDed), though I'm not sure I would see the point for 
this.

I really have stomach aches with the current design of the properties.


Ciao,
Michael.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-06 13:57     ` Michael Matz
@ 2019-03-07  6:56       ` Cary Coutant
  0 siblings, 0 replies; 11+ messages in thread
From: Cary Coutant @ 2019-03-07  6:56 UTC (permalink / raw)
  To: Michael Matz; +Cc: x86-64-abi, Binutils, Jim Dehnert

> > I was trying to understand why HJ didn't care about case (b) -- if a
> > feature is used by at least one object, what difference does it make
> > whether we have some objects that didn't say whether they used the
> > feature? I believe his answer is that the use case for these
> > properties is to allow the OS to decide whether it can allocate a
> > program to a downgraded processor, based on whether we know for sure
> > that the program does *not* use a particular feature. Thus, case (c)
> > is the important one where the downgrade is feasible, and all other
> > cases mean that it is not (either because we know the feature is used,
> > or because we do not know it's not used).
>
> That might be, but truth be told, I'm more comfortable with an orthogonal
> design that can express more than we need right now, than with a design
> where two concepts (known and used) are merged in a non-natural way.

Agreed.

> HJ: what's your issue with the KNOWN+USED approach?  I would envision an
> encoding where e.g. even IDs encode the 32 KNOWN bits, and odd IDs encode
> the 32 USED bits.  Optionally we could retain the differentiation between
> OR vs. AND for the logical combination of the USED bits (where the KNOWN
> bits are always ANDed), though I'm not sure I would see the point for
> this.

There are other properties that need AND: ones where the OS depends on
the program supporting something, like IBT and SHSTK, rather than the
program depending on a feature supported by OS or CPU. So they
wouldn't always be paired as KNOWN/USED, there would also be some AND
properties standing on their own. In addition, if we want NEEDED
features in addition to USED features, they could form a triplet:
KNOWN/USED/NEEDED. Still, I think using the even/odd bit as an
indication whether we need to OR or AND is reasonable (kind of like
how we use that bit to indicate what kind of tag is used for DT
entries). If we can eliminate the need for the OR_AND beast, I think
we could assert that the only combining methods we need are OR and
AND.

(I went back and looked at one of HJ's early proposals for properties,
and the even/odd bit is the approach he originally took.)

> I really have stomach aches with the current design of the properties.

As you can tell from my posts on this topic, I agree strongly enough
to have tilted at this windmill far longer than I normally would have.
Thanks for chiming in!

-cary

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-04 17:40 ` OR_AND semantics (was: GNU property saga) Michael Matz
  2019-03-05  0:48   ` Cary Coutant
@ 2019-03-08  3:44   ` H.J. Lu
  2019-03-08  6:06     ` Cary Coutant
  1 sibling, 1 reply; 11+ messages in thread
From: H.J. Lu @ 2019-03-08  3:44 UTC (permalink / raw)
  To: Michael Matz; +Cc: x86-64-abi, Binutils

On Tue, Mar 5, 2019 at 1:40 AM Michael Matz <matz@suse.de> wrote:
>
> Hi,
>
> On Mon, 4 Mar 2019, Michael Matz wrote:
>
> > Was there ever consensus on the OR_AND thingy?  It strikes me as not
> > really implementing what is wanted, especially in relation to future
> > extensibility (I'll write a mail about this).
>
> So, starting a subthread about only this.  The current description of the
> OR_AND range is this:
>
> ----------------
> \item[GNU_PROPERTY_X86_UINT32_OR_AND_LO..GNU_PROPERTY_X86_UINT32_OR_AND_HI]
>    A bit in the output \code{pr_data} field is set if it is set in any
>    relocatable input \code{pr_data} fields and this property is present
>    in all relocatable input files.  A missing property implies that its
>    bits have unknown values.  When all bits in the the output
>    \code{pr_data} field are zero, this property should not be removed
>    from output to indicate it has zero in all bits.  If the property
>    is in output, all input relocatables have the property.  If the bit
>    is 1, some input relocatables have the feature.  If the bit is 0,
> 
>    none of input relocatables have the feature.
> ----------------
>
> So, it's clear that the individual bits of the uint32 are independend of
> each other and hence can represent separate features/properties.  This
> means that the phrase "and this property is present in all relocatable
> input files" in not well defined: example situation:
>
> * producer A knows about bits 4 and 5 of FEATURE_X (assumed to be in the
>   OR_AND range), it will set the FEATURE_X property in a.o (and bit 4
>   and 5 therein to whatever is correct/requested)
> * producer B is older and happens to know only bit 4 of FEATURE_X, so it
>   will set FEATURE_X in b.o, and bit 4 to whatever value is correct, but
>   bit 5 will be zero.
>
> Now the above language would include FEATURE_X in the output, but bit 4
> would be reliable (set a.bit4 && b.bit4) while bit 5 would be unreliable
> (it is zero, even if it is one in a.o and _would_ be one in b.o had we
> used a newer producer).

We can't add a bit to FEATURE_X in such a way that  FEATURE_X generated
by older producers become invalid.   We must add the bit to FEATURE_Y which is
unsupported to all prior producers.  Am I missing something obvious?

> So there's still the confusion between absense and zero-is-unknown and
> zero-is-known, which the OR_AND range was designed to make clear (at least
> as far as I understood it from various mails trying to explain situations
> in which it would help).
>
> So, either I misunderstood the reason of existence for OR_AND, or there's
> some unspelled assumption in the usage of OR_AND (e.g. that there must
> never be producers knowing a different set of bits per FEATURE_xxx uint32
> set), or the language above doesn't capture the intent.
>
> What is it?
>

-- 
H.J.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-08  3:44   ` H.J. Lu
@ 2019-03-08  6:06     ` Cary Coutant
  2019-03-08  6:25       ` Jim Dehnert
  0 siblings, 1 reply; 11+ messages in thread
From: Cary Coutant @ 2019-03-08  6:06 UTC (permalink / raw)
  To: H.J. Lu; +Cc: Michael Matz, x86-64-abi, Binutils

> > Now the above language would include FEATURE_X in the output, but bit 4
> > would be reliable (set a.bit4 && b.bit4) while bit 5 would be unreliable
> > (it is zero, even if it is one in a.o and _would_ be one in b.o had we
> > used a newer producer).
>
> We can't add a bit to FEATURE_X in such a way that  FEATURE_X generated
> by older producers become invalid.   We must add the bit to FEATURE_Y which is
> unsupported to all prior producers.  Am I missing something obvious?

Yes, I think so. This is the point Michael (and I) were making. By
using KNOWN bits, you can add new feature bits without having to start
a whole new word of bits.

-cary

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-08  6:06     ` Cary Coutant
@ 2019-03-08  6:25       ` Jim Dehnert
  2019-03-11  7:34         ` H.J. Lu
  0 siblings, 1 reply; 11+ messages in thread
From: Jim Dehnert @ 2019-03-08  6:25 UTC (permalink / raw)
  To: Cary Coutant; +Cc: H.J. Lu, Michael Matz, x86-64-abi, Binutils

On Thu, Mar 7, 2019 at 10:06 PM Cary Coutant <ccoutant@gmail.com> wrote:

> > > Now the above language would include FEATURE_X in the output, but bit 4
> > > would be reliable (set a.bit4 && b.bit4) while bit 5 would be
> unreliable
> > > (it is zero, even if it is one in a.o and _would_ be one in b.o had we
> > > used a newer producer).
> >
> > We can't add a bit to FEATURE_X in such a way that  FEATURE_X generated
> > by older producers become invalid.   We must add the bit to FEATURE_Y
> which is
> > unsupported to all prior producers.  Am I missing something obvious?
>
> Yes, I think so. This is the point Michael (and I) were making. By
> using KNOWN bits, you can add new feature bits without having to start
> a whole new word of bits.
>

You also don't need to assume that a producer that has implemented support
for the tenth feature bit is also generating valid values for the
previously-defined nine, just because the words containing them are present.

Jim

>
> -cary
>
> --
> You received this message because you are subscribed to the Google Groups
> "X86-64 System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to x86-64-abi+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>


-- 
--
             Jim Dehnert
             dehnert@gmail.com

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: OR_AND semantics (was: GNU property saga)
  2019-03-08  6:25       ` Jim Dehnert
@ 2019-03-11  7:34         ` H.J. Lu
  0 siblings, 0 replies; 11+ messages in thread
From: H.J. Lu @ 2019-03-11  7:34 UTC (permalink / raw)
  To: Jim Dehnert; +Cc: Cary Coutant, Michael Matz, x86-64-abi, Binutils

On Fri, Mar 8, 2019 at 2:25 PM Jim Dehnert <dehnert@gmail.com> wrote:
>
>
>
> On Thu, Mar 7, 2019 at 10:06 PM Cary Coutant <ccoutant@gmail.com> wrote:
>>
>> > > Now the above language would include FEATURE_X in the output, but bit 4
>> > > would be reliable (set a.bit4 && b.bit4) while bit 5 would be unreliable
>> > > (it is zero, even if it is one in a.o and _would_ be one in b.o had we
>> > > used a newer producer).
>> >
>> > We can't add a bit to FEATURE_X in such a way that  FEATURE_X generated
>> > by older producers become invalid.   We must add the bit to FEATURE_Y which is
>> > unsupported to all prior producers.  Am I missing something obvious?
>>
>> Yes, I think so. This is the point Michael (and I) were making. By
>> using KNOWN bits, you can add new feature bits without having to start
>> a whole new word of bits.

So is this the only difference?  We don't always need to add a new set of
feature bits for a new feature if the new new feature will never be valid for
older producers like a new ISA bit.  But a new bit for the old ISA needs a
new set of bits.  This way, we only need to lookup one property for the
feature X instead of 2.

BTW, OR covers (b) in

(a) KNOWN = 0, USED = 0: The feature is not known to be used, but not
all objects provided a value.

(b) KNOWN = 0, USED = 1: The feature is used by at least one object,
but not all objects provided a value.

(c) KNOWN = 1, USED = 0: The feature is not used in any object, and
all objects provided a value.

(d) KNOWN = 1, USED = 1: The feature is used by at least one object,
and all objects provided a value.

since there is no functional difference for the feature X in output
object between

1. The feature is used by at least one object, but not all objects
provided a value.
2. The feature is used by at least one object, and all objects provided a value.

We only need to know if the feature X is used.   We do need "all
objects provided
a value" info for USED == 0 case.

>
> You also don't need to assume that a producer that has implemented support for the tenth feature bit is also generating valid values for the previously-defined nine, just because the words containing them are present.
>


-- 
H.J.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: GNU property saga
  2019-03-04 17:25 GNU property saga Michael Matz
  2019-03-04 17:40 ` OR_AND semantics (was: GNU property saga) Michael Matz
@ 2019-03-20 14:51 ` Mark Wielaard
       [not found]   ` <25adbffa-fc4c-1b01-7949-fbe0dc212f70@arm.com>
  1 sibling, 1 reply; 11+ messages in thread
From: Mark Wielaard @ 2019-03-20 14:51 UTC (permalink / raw)
  To: Michael Matz, x86-64-abi; +Cc: binutils

Hi,

On Mon, 2019-03-04 at 17:25 +0000, Michael Matz wrote:
> I'll admit that I have lost track of the consensus about GNU
> properties and all the discussions last year.  But we want to
> somewhen get this into the psABI document.

Thanks for picking this up. These things have a way of creeping up in
the various code bases without ever really reaching consensus. Causing
sad faces everywhere. Lets see if we can agree on something.

As far as I understand we did reach (a somewhat forced) consensus on
how to represent the IBT and Shadow Stack and Stack Size support which
the current glibc dynamic loader depends on.

The consensus on representation of those properties is described here:
https://sourceware.org/ml/binutils/2018-09/msg00282.html

But there doesn't seem to be any consensus anyon other/new GNU property
features or notes that would require the new (and/or) semantics.
bintuils and glibc don't even agree on the names and values of these
new feature properties.

You already split the discussion of the and/or semantics of those new
property features off into another thread. I think the answer to the
questions you pose below should also be split for the existing GNU
Properties (IBT/SHST/STACK_SIZE) and the new property/note features.

>   I believe there were two points (with subitems) of contentions:
>
> 1) use PT_NOTE vs PT_xxx for program header containing the properties
> 
> I believe the consensus was to use a new PT_xxx value

As far as I understand the current consensus, for the existing property
features, it is actually the opposite. There will be a new PT_NOTE
segment with (for ELFCLASS64) larger alignment and padding. This is
what the current glibc dynamic loader depends on.

But for the new GNU Property features it does seem that people would
prefer to use a new PT_xxx value. There is a new constant in binutils:
#define PT_GNU_PROPERTY	(PT_LOOS + 0x474e553) /* GNU property */
But that isn't used anywhere else.

Question is, if we are going to use that for new GNU Property features,
should we also do that for the old ones and how does that impact the
existing binaries/dynamic libraries?

> 1a) use new SHT_xxx for sections contain such properties
> 
> I believe the consensus was to stay with SHT_NOTE, even though that
> is squarely against ELF spirit.

Yes, for the existing GNU Properties. But I think it would be a bad
idea for the new property features if those will use a new PT value
segment type. Then the rules for composing SHT_NOTEs become even more
complex. You already need to distinguish between existing (GNU) notes
which don't combine with the new (Property) notes because they use
different alignment and padding rules. Then you would also get even
different notes that don't combine with any of the existing ones. At
least, the rules for combining them are not clear to me.

Cheers,

Mark

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: GNU property saga
       [not found]   ` <25adbffa-fc4c-1b01-7949-fbe0dc212f70@arm.com>
@ 2019-05-10 18:37     ` Mark Wielaard
  0 siblings, 0 replies; 11+ messages in thread
From: Mark Wielaard @ 2019-05-10 18:37 UTC (permalink / raw)
  To: Szabolcs Nagy, Michael Matz, x86-64-abi; +Cc: nd, binutils

On Tue, 2019-05-07 at 15:29 +0000, Szabolcs Nagy wrote:
> On 20/03/2019 14:51, Mark Wielaard wrote:
> > As far as I understand the current consensus, for the existing property
> > features, it is actually the opposite. There will be a new PT_NOTE
> > segment with (for ELFCLASS64) larger alignment and padding. This is
> > what the current glibc dynamic loader depends on.
> > 
> > But for the new GNU Property features it does seem that people would
> > prefer to use a new PT_xxx value. There is a new constant in binutils:
> > #define PT_GNU_PROPERTY	(PT_LOOS + 0x474e553) /* GNU property */
> > But that isn't used anywhere else.
> > 
> > Question is, if we are going to use that for new GNU Property features,
> > should we also do that for the old ones and how does that impact the
> > existing binaries/dynamic libraries?
> 
> what is a "new gnu property feature"?

The original GNU Properties were just for Control Flow Enforcement
(STACK_SIZE, NO_COPY_ON_PROTECTED and X86_FEATUREs IBT and SHSTK).
Those are in a special aligned SHT_NOTE section and PT_NOTE segment
with adjusted padding. These are recognized by the glibc dynamic loader
and some other tools.

There are also some new experimental properties, the X86_ISA Needed and
Used cpu instruction sets. For these new properties there doesn't seem
to be any consensus. binutils and glibc define some of them with
similar (but not always equal) GNU_PROPERTY names, but different
constants. There is discussion how the used and needed bits should be
interpreted (what if there is no such flag), see the other thread for a
discussion of that. And binutils seems to put them in a new
PT_GNU_PROPERTY segment which isn't recognized by anything else.

> if PT_GNU_PROPERTY is preferred, then aarch64 should use that.
> (aarch64 now uses gnu properties as well, but it does not have
> legacy binaries yet to be compatible with, only code in bfd
> that can be still fixed.)
> 
> but if current elf loaders/linkers only support PT_NOTE then
> switching to PT_GNU_PROPERTY may be messy (?)

Indeed. We first need consensus on all the issues above.

> > > 1a) use new SHT_xxx for sections contain such properties
> > > 
> > > I believe the consensus was to stay with SHT_NOTE, even though that
> > > is squarely against ELF spirit.
> > 
> > Yes, for the existing GNU Properties. But I think it would be a bad
> > idea for the new property features if those will use a new PT value
> > segment type. Then the rules for composing SHT_NOTEs become even more
> > complex. You already need to distinguish between existing (GNU) notes
> > which don't combine with the new (Property) notes because they use
> > different alignment and padding rules. Then you would also get even
> > different notes that don't combine with any of the existing ones. At
> > least, the rules for combining them are not clear to me.
> 
> that tells me PT_NOTE and SHT_NOTE for gnu properties
> should be left as they are today?

Yes, I think it would be best to keep with that for the original (CET
control flow related) GNU Properties.

But maybe for new ISA/cpuset used/needed instruction set properties we
should keep them separate and just start over and define a new
SHT_GNU_PROPERTIES and PT_GNU_PROPERTIES format.

Cheers,

Mark

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2019-05-10 18:37 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-04 17:25 GNU property saga Michael Matz
2019-03-04 17:40 ` OR_AND semantics (was: GNU property saga) Michael Matz
2019-03-05  0:48   ` Cary Coutant
2019-03-06 13:57     ` Michael Matz
2019-03-07  6:56       ` Cary Coutant
2019-03-08  3:44   ` H.J. Lu
2019-03-08  6:06     ` Cary Coutant
2019-03-08  6:25       ` Jim Dehnert
2019-03-11  7:34         ` H.J. Lu
2019-03-20 14:51 ` GNU property saga Mark Wielaard
     [not found]   ` <25adbffa-fc4c-1b01-7949-fbe0dc212f70@arm.com>
2019-05-10 18:37     ` Mark Wielaard

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).