OSABI on Linux Distros

OSABI on Linux Distros

[Suprateeka R Hegde]

Hi

What is the reason why we emit OSABI value as "UNIX - System V" 
(ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU" 
(ELFOSABI_GNU)?

I checked some major GNU/Linux distros like RHEL, SuseEL, Ubuntu, etc. 
Even though binaries use GNU extensions and ABI, the ELF is still marked 
as SYSV/None. Even a "core" file shows SYSV/None.

--
Supra

Re: OSABI on Linux Distros

[Roland McGrath]

> What is the reason why we emit OSABI value as "UNIX - System V" 
> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU" 
> (ELFOSABI_GNU)?

In GNU systems the use of this field is specifically to indicate that
certain GNU extensions to ELF are used in the particular object.  In
particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
then e_ident[EI_OSABI] must be ELFOSABI_GNU.  Otherwise, it can be
ELFOSABI_NONE (aka ELFOSABI_SYSV).  In fact, it should be, and the
tools (linkers) implement that distinction.  This makes it possible
for current tools (without special switches or whatnot) to produce
binaries that are compatible with older dynamic linker implementations
that did not support these GNU extensions to ELF.

This is arguably a bad overloading of the original instead of
EI_OSABI, but it's what's been done for GNU systems and so now it's an
unchangeable part of the GNU ELF gABI.

Re: OSABI on Linux Distros

[Carlos O'Donell]

On 06/10/2016 05:22 PM, Roland McGrath wrote:
>> What is the reason why we emit OSABI value as "UNIX - System V" 
>> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU" 
>> (ELFOSABI_GNU)?
> 
> In GNU systems the use of this field is specifically to indicate that
> certain GNU extensions to ELF are used in the particular object.  In
> particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
> then e_ident[EI_OSABI] must be ELFOSABI_GNU.  Otherwise, it can be
> ELFOSABI_NONE (aka ELFOSABI_SYSV).  In fact, it should be, and the
> tools (linkers) implement that distinction.  This makes it possible
> for current tools (without special switches or whatnot) to produce
> binaries that are compatible with older dynamic linker implementations
> that did not support these GNU extensions to ELF.
> 
> This is arguably a bad overloading of the original instead of
> EI_OSABI, but it's what's been done for GNU systems and so now it's an
> unchangeable part of the GNU ELF gABI.
 
Sure we might have to document this in the GNU ELF gABI, but nothing
stops us from changing it and moving forward?

Change binutils to generate ELFOSABI_GNU by default, with a 
"special switch" to allow people to generate pre-2009 glibc ld.so
compatible binaries if they need it. You would need a sysroot to build 
with so they'd be using probably similar vintage tools anyway and don't
have to worry. Then all binaries going forward are marked as being
ELFOSABI_GNU binaries.

The use case of being able to use ELFOSABI_GNU to determine how to decode
values between PT_LOOS/PT_HIOS is the most relevant I've seen.

Today with ELFOSABI_NONE you don't know how to decode those values, only
though if one assumes that PT_LOOS/PT_HIOS values can overlap between
any OS.

-- 
Cheers,
Carlos.

Re: OSABI on Linux Distros

[Suprateeka R Hegde]

On 18-Jun-2016 10:41 AM, Carlos O'Donell wrote:
> On 06/10/2016 05:22 PM, Roland McGrath wrote:
>>> What is the reason why we emit OSABI value as "UNIX - System V"
>>> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU"
>>> (ELFOSABI_GNU)?
>>
>> In GNU systems the use of this field is specifically to indicate that
>> certain GNU extensions to ELF are used in the particular object.  In
>> particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
>> then e_ident[EI_OSABI] must be ELFOSABI_GNU.  Otherwise, it can be
>> ELFOSABI_NONE (aka ELFOSABI_SYSV).  In fact, it should be, and the
>> tools (linkers) implement that distinction.  This makes it possible
>> for current tools (without special switches or whatnot) to produce
>> binaries that are compatible with older dynamic linker implementations
>> that did not support these GNU extensions to ELF.
>>
>> This is arguably a bad overloading of the original instead of
>> EI_OSABI, but it's what's been done for GNU systems and so now it's an
>> unchangeable part of the GNU ELF gABI.
>
> Sure we might have to document this in the GNU ELF gABI, but nothing
> stops us from changing it and moving forward?
>
> Change binutils to generate ELFOSABI_GNU by default, with a
> "special switch" to allow people to generate pre-2009 glibc ld.so
> compatible binaries if they need it. You would need a sysroot to build
> with so they'd be using probably similar vintage tools anyway and don't
> have to worry. Then all binaries going forward are marked as being
> ELFOSABI_GNU binaries.
>
> The use case of being able to use ELFOSABI_GNU to determine how to decode
> values between PT_LOOS/PT_HIOS is the most relevant I've seen.
>
> Today with ELFOSABI_NONE you don't know how to decode those values, only
> though if one assumes that PT_LOOS/PT_HIOS values can overlap between
> any OS.

Thats absolutely what I think. I have a patch. I shall post it here soon.

And if we have good consensus, I shall talk to H.J to get this added to 
GNU-gABI.

--
Supra

Re: OSABI on Linux Distros

[Suprateeka R Hegde]

On 11-Jun-2016 02:52 AM, Roland McGrath wrote:
>> What is the reason why we emit OSABI value as "UNIX - System V"
>> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU"
>> (ELFOSABI_GNU)?
>
> In GNU systems the use of this field is specifically to indicate that
> certain GNU extensions to ELF are used in the particular object.  In
> particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
> then e_ident[EI_OSABI] must be ELFOSABI_GNU.

OK. I understood this part. But then I did not know this set is partial. 
I thought use of *any* GNU specific extension should mark it as GNU OSABI.

For instance, GNU_RELRO, GNU_EH_FRAME, GNU_STACK, etc. Even with all 
these, it is still marked SYSV ABI.

So,

1. Do we have a documented list of GNU extensions that are necessary to 
mark an ELF as GNU ABI?
2. Why the list is partial? Why not all GNU extensions?

--
Supra

   Otherwise, it can be
> ELFOSABI_NONE (aka ELFOSABI_SYSV).  In fact, it should be, and the
> tools (linkers) implement that distinction.  This makes it possible
> for current tools (without special switches or whatnot) to produce
> binaries that are compatible with older dynamic linker implementations
> that did not support these GNU extensions to ELF.
>
> This is arguably a bad overloading of the original instead of
> EI_OSABI, but it's what's been done for GNU systems and so now it's an
> unchangeable part of the GNU ELF gABI.
>

Re: OSABI on Linux Distros

[Carlos O'Donell]

On 06/11/2016 02:44 AM, Suprateeka R Hegde wrote:
> On 11-Jun-2016 02:52 AM, Roland McGrath wrote:
>>> What is the reason why we emit OSABI value as "UNIX - System V"
>>> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU"
>>> (ELFOSABI_GNU)?
>>
>> In GNU systems the use of this field is specifically to indicate that
>> certain GNU extensions to ELF are used in the particular object.  In
>> particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
>> then e_ident[EI_OSABI] must be ELFOSABI_GNU.
> 
> OK. I understood this part. But then I did not know this set is
> partial. I thought use of *any* GNU specific extension should mark it
> as GNU OSABI.
> 
> For instance, GNU_RELRO, GNU_EH_FRAME, GNU_STACK, etc. Even with all
> these, it is still marked SYSV ABI.

I'm not as authoritative on this as Roland is, but in my opinion I expect
that because all three of these entries are optional they do not constitute
a change in the EI_OSABI.

You can ignore GNU_RELRO without any problem, you just won't have a
read-only segment after relocation.

You can ignore GNU_STACK without any problems, it's information
for the dynamic loader to set default stack permissions. When unset the
glibc dynamic loader will use sensible per-machine defaults, likewise
other implementations should also.

The GNU_EH_FRAME can be ignored if you're not using the GNU-based
unwinder which uses this PT_* entry to find .eh_frame_hdr.

Other dynamic linkers should ignore program header segments they don't
understand, and thus the above optionally processed entries should not
change the EI_OSABI.

> 1. Do we have a documented list of GNU extensions that are necessary
> to mark an ELF as GNU ABI?

Not that I am aware of. H.J. might better answer this question.

> 2. Why the list is partial? Why not all GNU extensions?

Only the required GNU extensions should mark EI_OSABI as ELFOSABI_GNU.

If you don't implement STT_GNU_IFUNC or STB_GNU_UNIQUE then things
actually fail.

-- 
Cheers,
Carlos.

Re: OSABI on Linux Distros

[Suprateeka R Hegde]

On 13-Jun-2016 08:56 PM, Carlos O'Donell wrote:
> On 06/11/2016 02:44 AM, Suprateeka R Hegde wrote:
>> On 11-Jun-2016 02:52 AM, Roland McGrath wrote:
>>>> What is the reason why we emit OSABI value as "UNIX - System V"
>>>> (ELFOSABI_SYSV) even on GNU/Linux systems? Shouldn't it be just "GNU"
>>>> (ELFOSABI_GNU)?
>>>
>>> In GNU systems the use of this field is specifically to indicate that
>>> certain GNU extensions to ELF are used in the particular object.  In
>>> particular, if any .dynsym entries use STT_GNU_IFUNC o STB_GNU_UNIQUE,
>>> then e_ident[EI_OSABI] must be ELFOSABI_GNU.
>>
>> OK. I understood this part. But then I did not know this set is
>> partial. I thought use of *any* GNU specific extension should mark it
>> as GNU OSABI.
>>
>> For instance, GNU_RELRO, GNU_EH_FRAME, GNU_STACK, etc. Even with all
>> these, it is still marked SYSV ABI.
>
> I'm not as authoritative on this as Roland is, but in my opinion I expect
> that because all three of these entries are optional they do not constitute
> a change in the EI_OSABI.
>
> You can ignore GNU_RELRO without any problem, you just won't have a
> read-only segment after relocation.
>
> You can ignore GNU_STACK without any problems, it's information
> for the dynamic loader to set default stack permissions. When unset the
> glibc dynamic loader will use sensible per-machine defaults, likewise
> other implementations should also.
>
> The GNU_EH_FRAME can be ignored if you're not using the GNU-based
> unwinder which uses this PT_* entry to find .eh_frame_hdr.
>
> Other dynamic linkers should ignore program header segments they don't
> understand, and thus the above optionally processed entries should not
> change the EI_OSABI.

You are considering only dynamic linker here.

>
>> 1. Do we have a documented list of GNU extensions that are necessary
>> to mark an ELF as GNU ABI?
>
> Not that I am aware of. H.J. might better answer this question.

I shall wait for the answer.

>
>> 2. Why the list is partial? Why not all GNU extensions?
>
> Only the required GNU extensions should mark EI_OSABI as ELFOSABI_GNU.
>
> If you don't implement STT_GNU_IFUNC or STB_GNU_UNIQUE then things
> actually fail.

It depends on what you are considering here. You considered only dynamic 
linker. I consider even non-GNU tools that reads ELF. When such a tool 
encounters SYSV as the ABI, it cannot do much. However, if the tool can 
see that the ELF has GNU extensions, then it can do better job of 
dumping GNU specific ELF details.

My actual question is what is the harm if we mark GNU ABI for *any* GNU 
extension and not restrict it to a partial list.

--
Supra

Re: OSABI on Linux Distros

[Carlos O'Donell]

On 06/13/2016 11:57 AM, Suprateeka R Hegde wrote:
> It depends on what you are considering here. You considered only
> dynamic linker. I consider even non-GNU tools that reads ELF. When
> such a tool encounters SYSV as the ABI, it cannot do much. However,
> if the tool can see that the ELF has GNU extensions, then it can do
> better job of dumping GNU specific ELF details.
> 
> My actual question is what is the harm if we mark GNU ABI for *any*
> GNU extension and not restrict it to a partial list.

That's a much better question, but still doesn't yet get at the crux
of your argument.

It might be said that it should be possible to identify the OS from
the OSABI value in order to aid in the processing of PT_LOOS to
PT_HIOS entries (GNU_EH_FRAME, GNU_STACK, GNU_RELRO)?

Is that what you are thinking?

I expect that entries between PT_LOOS to PT_HIOS are allowed to
overlap between OSs?

-- 
Cheers,
Carlos.

Re: OSABI on Linux Distros

[Suprateeka R Hegde]

On 13-Jun-2016 11:30 PM, Carlos O'Donell wrote:
> On 06/13/2016 11:57 AM, Suprateeka R Hegde wrote:
>> It depends on what you are considering here. You considered only
>> dynamic linker. I consider even non-GNU tools that reads ELF. When
>> such a tool encounters SYSV as the ABI, it cannot do much. However,
>> if the tool can see that the ELF has GNU extensions, then it can do
>> better job of dumping GNU specific ELF details.
>>
>> My actual question is what is the harm if we mark GNU ABI for *any*
>> GNU extension and not restrict it to a partial list.
>
> That's a much better question, but still doesn't yet get at the crux
> of your argument.

I thought the use cases are implicit. Sorry for the ambiguity.

>
> It might be said that it should be possible to identify the OS from
> the OSABI value in order to aid in the processing of PT_LOOS to
> PT_HIOS entries (GNU_EH_FRAME, GNU_STACK, GNU_RELRO)?
>
> Is that what you are thinking?

Yes. But not just to process PT_LOOS-PT_HIOS. On certain enterprise 
deployments there are necessities to sort or edit ELFs based on OS. 
OSABI is one of the quickest and safest/reliable way. There are more 
such use cases.

> I expect that entries between PT_LOOS to PT_HIOS are allowed to
> overlap between OSs?

Yes thats right. If they had to be unique, they could well be part of 
gABI itself.

--
Supra