* binutils ld and new PT_GNU_PROPERTY segment @ 2020-01-01 0:00 Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 0 siblings, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: gnu-gabi, binutils Hi, binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553) segment that overlaps with the PT_NOTE segment covering the .note.gnu.property section data. I cannot tell if this is an accident/experiment that happened to end up in a release or if it is proposed as an official new segment type. As far as I can tell binutils ld is the only linker which adds this extra segment and there is no runtime loader which uses it. It doesn't provide any new information that cannot be found in the existing PT_NOTE segment. On 64 bit architectures it simply covers the extra existing PT_NOTE with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On 32bit architectures it covers a sub-range of the existing PT_NOTE segment. It isn't clear to me how other tools should handle this. It seems to prevent normal merging of note sections. Since some notes are probably special if they need to be covered by this new segment type. And it isn't clear how the linker knows which of the SHT_NOTE sections is what needs to be covered by the new segment type. Or is the idea that this will eventually come with a new section type too and GNU properties will no longer use NOTE sections? Thanks, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 binutils ld and new PT_GNU_PROPERTY segment Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Fangrui Song via gnu-gabi 2020-01-01 0:00 ` Mark Wielaard 0 siblings, 2 replies; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard; +Cc: Rahul Chaudhry via gnu-gabi, Binutils, Zhang, Annita On Tue, Feb 18, 2020 at 4:38 AM Mark Wielaard <mark@klomp.org> wrote: > > Hi, > > binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553) > segment that overlaps with the PT_NOTE segment covering the > .note.gnu.property section data. > > I cannot tell if this is an accident/experiment that happened to end up > in a release or if it is proposed as an official new segment type. https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI > As far as I can tell binutils ld is the only linker which adds this Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not, it is an lld bug. > extra segment and there is no runtime loader which uses it. It doesn't > provide any new information that cannot be found in the existing > PT_NOTE segment. Kernel loader uses it for both ARM and x86. > On 64 bit architectures it simply covers the extra existing PT_NOTE > with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On > 32bit architectures it covers a sub-range of the existing PT_NOTE > segment. > > It isn't clear to me how other tools should handle this. It seems to > prevent normal merging of note sections. Since some notes are probably > special if they need to be covered by this new segment type. And it > isn't clear how the linker knows which of the SHT_NOTE sections is what > needs to be covered by the new segment type. Or is the idea that this > will eventually come with a new section type too and GNU properties > will no longer use NOTE sections? > PT_GNU_PROPERTY covers .note.gnu.property section. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Fangrui Song via gnu-gabi 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` Zhang, Annita 2020-01-01 0:00 ` Mark Wielaard 1 sibling, 2 replies; 27+ messages in thread From: Fangrui Song via gnu-gabi @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu, Zhang, Annita; +Cc: Mark Wielaard, gnu-gabi, Binutils On 2020-02-18, H.J. Lu wrote: >On Tue, Feb 18, 2020@4:38 AM Mark Wielaard <mark@klomp.org> wrote: >> >> Hi, >> >> binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553) >> segment that overlaps with the PT_NOTE segment covering the >> .note.gnu.property section data. >> >> I cannot tell if this is an accident/experiment that happened to end up >> in a release or if it is proposed as an official new segment type. > >https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html >https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI > >> As far as I can tell binutils ld is the only linker which adds this > >Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not, >it is an lld bug. > >> extra segment and there is no runtime loader which uses it. It doesn't >> provide any new information that cannot be found in the existing >> PT_NOTE segment. > >Kernel loader uses it for both ARM and x86. > >> On 64 bit architectures it simply covers the extra existing PT_NOTE >> with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On >> 32bit architectures it covers a sub-range of the existing PT_NOTE >> segment. >> >> It isn't clear to me how other tools should handle this. It seems to >> prevent normal merging of note sections. Since some notes are probably >> special if they need to be covered by this new segment type. And it >> isn't clear how the linker knows which of the SHT_NOTE sections is what >> needs to be covered by the new segment type. Or is the idea that this >> will eventually come with a new section type too and GNU properties >> will no longer use NOTE sections? >> > >PT_GNU_PROPERTY covers .note.gnu.property section. https://reviews.llvm.org/D70961 added PT_GNU_PROPERTY support to lld. The change will be included by lld 10.0.0 (currently at rc2). https://reviews.llvm.org/D70959 added the dump support to llvm-readelf -l and llvm-objdump -p. The change will be included by LLVM 10.0.0 (currently at rc2). From what I can see, neither the Linux kernel nor glibc uses PT_GNU_PROPERTY. glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. I tend to agree with Cary (https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that .note.gnu.property should have been designed as a different section type because its combining semantics are different from other notes (we could apply "Rules for Linking Unrecognized Sections" to all SHT_NOTE sections) but it is too late to change the section type. A separate segment type (PT_GNU_PROPERTY) looks fine to me. glibc should probably be updated to parse PT_GNU_PROPERTY instead. (Recently I read some ABI decisions and I noticed that I frequently see the term "it is too late". As a contributor of both lld and LLVM binary utilities (and the implementer of a bunch of GNU_PROPERTY changes), I hope that the LLVM community can be informed of such changes earlier. A lot of people are not subscribed to any of the mailing lists (recently I visit the archives from time to time). Looks like Annita's job? :) ) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Fangrui Song via gnu-gabi @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Zhang, Annita 1 sibling, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: Fangrui Song, H.J. Lu, Zhang, Annita; +Cc: gnu-gabi, Binutils Hi, On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote: > From what I can see, neither the Linux kernel nor glibc uses > PT_GNU_PROPERTY. > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. That is my impression too. > I tend to agree with Cary > (https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that > .note.gnu.property should have been designed as a different section > type > because its combining semantics are different from other notes > (we could apply "Rules for Linking Unrecognized Sections" to all > SHT_NOTE sections) but it is too late to change the section type. Agreed. > A separate segment type (PT_GNU_PROPERTY) looks fine to me. > glibc should probably be updated to parse PT_GNU_PROPERTY instead. I think it is confusing to now introduce a new segment type which basically provides the same information as the PT_NOTE segement. It is ill defined because it needs to be matched to a magic section name. Which makes things harder for tools dealing with generic SHT_NOTE sections (they would have to preserve the magic section name, might not be able to merge notes, etc.) > (Recently I read some ABI decisions and I noticed that I frequently see > the term "it is too late". As a contributor of both lld and LLVM binary > utilities (and the implementer of a bunch of GNU_PROPERTY changes), I > hope that the LLVM community can be informed of such changes earlier. A > lot of people are not subscribed to any of the mailing lists (recently I > visit the archives from time to time). Yeah, we really need to do a better job coordinating and communicating. Even though I am trying to keep up with these lists I am often surprised by changes like this which seem to have just been added to binutils without trying to get consensus with other GNU/ELF tool implementations about the precise semantics or whether a feature is simply redundant with existing practice. Cheers, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <mark@klomp.org> wrote: > > Hi, > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote: > > From what I can see, neither the Linux kernel nor glibc uses > > PT_GNU_PROPERTY. > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. > > That is my impression too. See: https://patchwork.kernel.org/patch/11285409/ It is for both x86 and arm64. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 0 siblings, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils Hi, On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote: > On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <mark@klomp.org> wrote: > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote: > > > From what I can see, neither the Linux kernel nor glibc uses > > > PT_GNU_PROPERTY. > > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. > > > > That is my impression too. > > See: > > https://patchwork.kernel.org/patch/11285409/ > > It is for both x86 and arm64. So that is not upstream in the mainline kernel? Why can't that patch use the existing PT_NOTE segment? That would make it compatible with existing binaries that don't have this PT_GNU_PROPERTY program header. Thanks, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 0 siblings, 2 replies; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > > Hi, > > On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote: > > On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <mark@klomp.org> wrote: > > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote: > > > > From what I can see, neither the Linux kernel nor glibc uses > > > > PT_GNU_PROPERTY. > > > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. > > > > > > That is my impression too. > > > > See: > > > > https://patchwork.kernel.org/patch/11285409/ > > > > It is for both x86 and arm64. > > So that is not upstream in the mainline kernel? Why can't that patch > use the existing PT_NOTE segment? That would make it compatible with > existing binaries that don't have this PT_GNU_PROPERTY program header. Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader only wants to check PT_XXX. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 1 sibling, 0 replies; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 4:28 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > > > > Hi, > > > > On Wed, 2020-02-19 at 03:39 -0800, H.J. Lu wrote: > > > On Wed, Feb 19, 2020 at 2:58 AM Mark Wielaard <mark@klomp.org> wrote: > > > > On Tue, 2020-02-18 at 18:31 -0800, Fangrui Song wrote: > > > > > From what I can see, neither the Linux kernel nor glibc uses > > > > > PT_GNU_PROPERTY. > > > > > glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. > > > > > > > > That is my impression too. > > > > > > See: > > > > > > https://patchwork.kernel.org/patch/11285409/ > > > > > > It is for both x86 and arm64. > > > > So that is not upstream in the mainline kernel? Why can't that patch > > use the existing PT_NOTE segment? That would make it compatible with > > existing binaries that don't have this PT_GNU_PROPERTY program header. > > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader > only wants to check PT_XXX. > This is one thread on kernel mailing list: https://lore.kernel.org/lkml/87imt4jwpt.fsf@oldenburg2.str.redhat.com/ -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 1 sibling, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > > > https://patchwork.kernel.org/patch/11285409/ > > > > > > It is for both x86 and arm64. > > > > So that is not upstream in the mainline kernel? Why can't that patch > > use the existing PT_NOTE segment? That would make it compatible with > > existing binaries that don't have this PT_GNU_PROPERTY program header. > > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader > only wants to check PT_XXX. So they can check PT_NOTE because it provides the same information and is already available in existing binaries. Cheers, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Fangrui Song 0 siblings, 1 reply; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard; +Cc: Fangrui Song, Zhang, Annita, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <mark@klomp.org> wrote: > > On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: > > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > > > > https://patchwork.kernel.org/patch/11285409/ > > > > > > > > It is for both x86 and arm64. > > > > > > So that is not upstream in the mainline kernel? Why can't that patch > > > use the existing PT_NOTE segment? That would make it compatible with > > > existing binaries that don't have this PT_GNU_PROPERTY program header. > > > > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader > > only wants to check PT_XXX. > > So they can check PT_NOTE because it provides the same information and > is already available in existing binaries. > Please take a look at glibc note.gnu.property parser. It is very complicated to check for invalid .note.gnu.property sections generated by the older linkers with the new object. Kernel loader doesn't want to do it. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Fangrui Song 2020-01-01 0:00 ` H.J. Lu 0 siblings, 1 reply; 27+ messages in thread From: Fangrui Song @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu Cc: Mark Wielaard, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On 2020-02-19, H.J. Lu wrote: >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <mark@klomp.org> wrote: >> >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: >> > > > https://patchwork.kernel.org/patch/11285409/ >> > > > >> > > > It is for both x86 and arm64. >> > > >> > > So that is not upstream in the mainline kernel? Why can't that patch >> > > use the existing PT_NOTE segment? That would make it compatible with >> > > existing binaries that don't have this PT_GNU_PROPERTY program header. >> > >> > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader >> > only wants to check PT_XXX. >> >> So they can check PT_NOTE because it provides the same information and >> is already available in existing binaries. >> > >Please take a look at glibc note.gnu.property parser. It is very complicated to >check for invalid .note.gnu.property sections generated by the older >linkers with >the new object. Kernel loader doesn't want to do it. One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html * Define SHT_GNU_PROPERTY * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. ( https://patchwork.kernel.org/patch/11285409/ needs no change) This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. I will be happy to make changes to lld and LLVM binary utilities if this scheme reaches consensus. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Fangrui Song @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Fangrui Song Cc: Mark Wielaard, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <i@maskray.me> wrote: > > On 2020-02-19, H.J. Lu wrote: > >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <mark@klomp.org> wrote: > >> > >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: > >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > >> > > > https://patchwork.kernel.org/patch/11285409/ > >> > > > > >> > > > It is for both x86 and arm64. > >> > > > >> > > So that is not upstream in the mainline kernel? Why can't that patch > >> > > use the existing PT_NOTE segment? That would make it compatible with > >> > > existing binaries that don't have this PT_GNU_PROPERTY program header. > >> > > >> > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader > >> > only wants to check PT_XXX. > >> > >> So they can check PT_NOTE because it provides the same information and > >> is already available in existing binaries. > >> > > > >Please take a look at glibc note.gnu.property parser. It is very complicated to > >check for invalid .note.gnu.property sections generated by the older > >linkers with > >the new object. Kernel loader doesn't want to do it. > > One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html > > * Define SHT_GNU_PROPERTY > * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY > * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment > > The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). > New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. > ( https://patchwork.kernel.org/patch/11285409/ needs no change) > > This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. > > I will be happy to make changes to lld and LLVM binary utilities if this > scheme reaches consensus. It is kind of too late now. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 0 siblings, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu, Fangrui Song Cc: Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils Hi, On Wed, 2020-02-19 at 11:29 -0800, H.J. Lu wrote: > On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <i@maskray.me> wrote: > > One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html > > > > * Define SHT_GNU_PROPERTY > > * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY > > * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment > > > > The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). > > New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. > > ( https://patchwork.kernel.org/patch/11285409/ needs no change) > > > > This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. > > > > I will be happy to make changes to lld and LLVM binary utilities if this > > scheme reaches consensus. > > It is kind of too late now. This code isn't in the kernel yet. So either it gets changed to use the existing scheme with gnu property notes found through PT_NOTE to work with existing binaries. Then there is no need for PT_GNU_PROPERTY headers. Or some future kernel will start using PT_GNU_PROPERTY headers to find the gnu property notes. But that means it won't work with existing binaries that do not have that header. So there is no backwards compatibility anyway and we can define SHT_GNU_PROPERTY like above. So this actually seems the perfect time to make this decision. Cheers, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 0 siblings, 1 reply; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard Cc: Fangrui Song, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <mark@klomp.org> wrote: > > Hi, > > On Wed, 2020-02-19 at 11:29 -0800, H.J. Lu wrote: > > On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <i@maskray.me> wrote: > > > One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html > > > > > > * Define SHT_GNU_PROPERTY > > > * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY > > > * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment > > > > > > The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). > > > New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. > > > ( https://patchwork.kernel.org/patch/11285409/ needs no change) > > > > > > This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. > > > > > > I will be happy to make changes to lld and LLVM binary utilities if this > > > scheme reaches consensus. > > > > It is kind of too late now. > > This code isn't in the kernel yet. So either it gets changed to use the > existing scheme with gnu property notes found through PT_NOTE to work > with existing binaries. Then there is no need for PT_GNU_PROPERTY > headers. > > Or some future kernel will start using PT_GNU_PROPERTY headers to find > the gnu property notes. But that means it won't work with existing > binaries that do not have that header. So there is no backwards > compatibility anyway and we can define SHT_GNU_PROPERTY like above. > > So this actually seems the perfect time to make this decision. > Binaries with .note.gnu.property section have been put into many OS releases. We must support them. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 0 siblings, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu Cc: Fangrui Song, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils Hi, On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote: > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <mark@klomp.org> wrote: > > This code isn't in the kernel yet. So either it gets changed to use the > > existing scheme with gnu property notes found through PT_NOTE to work > > with existing binaries. Then there is no need for PT_GNU_PROPERTY > > headers. > > > > Or some future kernel will start using PT_GNU_PROPERTY headers to find > > the gnu property notes. But that means it won't work with existing > > binaries that do not have that header. So there is no backwards > > compatibility anyway and we can define SHT_GNU_PROPERTY like above. > > > > So this actually seems the perfect time to make this decision. > > Binaries with .note.gnu.property section have been put into many > OS releases. We must support them. OK. Then it is option 1. The kernel will need to support PT_NOTE for parsing the properties, since such older binaries won't have a PT_GNU_PROPERTY program header. Then we can simply get rid of PT_GNU_PROPERTY since nobody uses it and all information is already available through the PT_NOTE segment. Cheers, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` Fangrui Song 0 siblings, 2 replies; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard Cc: Fangrui Song, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <mark@klomp.org> wrote: > > Hi, > > On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote: > > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <mark@klomp.org> wrote: > > > This code isn't in the kernel yet. So either it gets changed to use the > > > existing scheme with gnu property notes found through PT_NOTE to work > > > with existing binaries. Then there is no need for PT_GNU_PROPERTY > > > headers. > > > > > > Or some future kernel will start using PT_GNU_PROPERTY headers to find > > > the gnu property notes. But that means it won't work with existing > > > binaries that do not have that header. So there is no backwards > > > compatibility anyway and we can define SHT_GNU_PROPERTY like above. > > > > > > So this actually seems the perfect time to make this decision. > > > > Binaries with .note.gnu.property section have been put into many > > OS releases. We must support them. > > OK. Then it is option 1. The kernel will need to support PT_NOTE for > parsing the properties, since such older binaries won't have a > PT_GNU_PROPERTY program header. Then we can simply get rid of > PT_GNU_PROPERTY since nobody uses it and all information is already > available through the PT_NOTE segment. > Kernel loader only checks ld.so and static executable. Re-link them with newer linker will get PT_GNU_PROPERTY. But ld.so needs to check PT_NOTE for older binaries. -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu @ 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` Fangrui Song 2020-01-01 0:00 ` Fangrui Song 1 sibling, 1 reply; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu Cc: Fangrui Song, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils Hi, On Thu, 2020-02-20 at 03:51 -0800, H.J. Lu wrote: > On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <mark@klomp.org> wrote: > > On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote: > > > > > > Binaries with .note.gnu.property section have been put into many > > > OS releases. We must support them. > > > > OK. Then it is option 1. The kernel will need to support PT_NOTE for > > parsing the properties, since such older binaries won't have a > > PT_GNU_PROPERTY program header. Then we can simply get rid of > > PT_GNU_PROPERTY since nobody uses it and all information is already > > available through the PT_NOTE segment. > > > > Kernel loader only checks ld.so and static executable. Re-link them with > newer linker will get PT_GNU_PROPERTY. But ld.so needs to check > PT_NOTE for older binaries. Having different loaders check different program headers seems very confusing. And it doesn't really seem to provide backwards compatibility since depending on how the code was linked previously you might still require a rebuild. Having a mix of PT_NOTE/PT_GNU_PROPERTY segments both based on SHT_NOTEs, but one of them based on magic section names, makes things really complicated for other tools. Please lets either really keep things as they are or redesign things based on PT_GNU_PROPERTY and a new section type. Thanks, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` Fangrui Song 2020-04-01 8:46 ` Florian Weimer 0 siblings, 1 reply; 27+ messages in thread From: Fangrui Song @ 2020-01-01 0:00 UTC (permalink / raw) To: Mark Wielaard, H.J. Lu Cc: Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, GNU C Library, Binutils + libc-alpha On 2020-02-20, Mark Wielaard wrote: >Hi, > >On Thu, 2020-02-20@03:51 -0800, H.J. Lu wrote: >> On Thu, Feb 20, 2020@1:37 AM Mark Wielaard <mark@klomp.org> wrote: >> > On Wed, 2020-02-19@14:17 -0800, H.J. Lu wrote: >> > > >> > > Binaries with .note.gnu.property section have been put into many >> > > OS releases. We must support them. >> > >> > OK. Then it is option 1. The kernel will need to support PT_NOTE for >> > parsing the properties, since such older binaries won't have a >> > PT_GNU_PROPERTY program header. Then we can simply get rid of >> > PT_GNU_PROPERTY since nobody uses it and all information is already >> > available through the PT_NOTE segment. >> > >> >> Kernel loader only checks ld.so and static executable. Re-link them with >> newer linker will get PT_GNU_PROPERTY. But ld.so needs to check >> PT_NOTE for older binaries. > >Having different loaders check different program headers seems very >confusing. And it doesn't really seem to provide backwards >compatibility since depending on how the code was linked previously you >might still require a rebuild. > >Having a mix of PT_NOTE/PT_GNU_PROPERTY segments both based on >SHT_NOTEs, but one of them based on magic section names, makes things >really complicated for other tools. Agreed. The current (glibc: PT_NOTE; proposed Linux: PT_GNU_PROPERTY) mixed mess is looking for trouble. Unfortunately we are likely in a situation where we have to revise ABI[0] just because we did not try hard to get consensus[1][2]. >Please lets either really keep things as they are or redesign things >based on PT_GNU_PROPERTY and a new section type. > >Thanks, > >Mark Below is my understanding of these matters. Hope they will be useful for interested stakeholders (for example, AArch64 devs, though PT_GNU_PROPERTY is currently driven by x86) who don't follow the discussions so closely. 1. We need PT_GNU_PROPERTY. Old linkers don't know the special processing on input .note.gnu.property sections. The output .note.gnu.property does not take -z ibt/-z shstk/-z force-bti/-z pac-plt into account => invalid. The produced PT_NOTE may contain multiple NT_GNU_PROPERTY_TYPE_0 => invalid [3] Also note that sh_addralign(.note.gnu.property)=8 on a 64-bit platform, while sh_addralign(.note.gnu.build-id)=sh_addralign(.note.ABI-tag)=...=4 (ancient mistake made by at least Linux/FreeBSD/NetBSD/...) GNU ld before PR ld/23658 may create corrupted PT_NOTE. For at least the above reasons, loaders are better not interpreting PT_NOTE. glibc/sysdeps/x86/dl-prop.h is currently interpreting PT_NOTE => it should be fixed. Given point 1 and 3, this comment deserves a reconsideration: > Binaries with .note.gnu.property section have been put into many > OS releases. We must support them. 2. .note.gnu.property behaves strangely, unlike a regular SHT_NOTE. For a .note.gnu.property aware linker (newer GNU ld, newer lld), .note.gnu.property input sections are dropped. (We have .note.GNU-stack and .note.GNU-split-stack which both require special processing, but they are SHT_PROGBITS.) 3. We need SHT_GNU_PROPERTY. The output .note.gnu.property being SHT_NOTE causes linkers to place the section in both PT_NOTE and PT_GNU_PROPERTY. PT_NOTE, as explained by point 1 above, can cause trouble to old loaders. Have we proved that "older linker-produced concatenated PT_NOTE cannot cause trouble to loaders interpreting PT_NOTE"? SHT_GNU_PROPERTY does not contribute to PT_NOTE and will not cause any problem to old loaders interpreting PT_NOTE. [0]: https://sourceware.org/ml/gnu-gabi/2018-q4/msg00030.html [1]: https://sourceware.org/ml/gnu-gabi/2018-q4/msg00033.html [2]: https://sourceware.org/ml/gnu-gabi/2018-q4/msg00037.html [3]: https://groups.google.com/forum/#!topic/x86-64-abi/LGPTWbMp8hQ ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Fangrui Song @ 2020-04-01 8:46 ` Florian Weimer 2020-04-01 9:22 ` Szabolcs Nagy 0 siblings, 1 reply; 27+ messages in thread From: Florian Weimer @ 2020-04-01 8:46 UTC (permalink / raw) To: Fangrui Song Cc: Mark Wielaard, H.J. Lu, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, GNU C Library, Binutils * Fangrui Song: > Below is my understanding of these matters. Hope they will be useful for > interested stakeholders (for example, AArch64 devs, though PT_GNU_PROPERTY is > currently driven by x86) who don't follow the discussions so closely. > > 1. We need PT_GNU_PROPERTY. > Old linkers don't know the special processing on input .note.gnu.property sections. > The output .note.gnu.property does not take -z ibt/-z shstk/-z force-bti/-z pac-plt into account => > invalid. > The produced PT_NOTE may contain multiple NT_GNU_PROPERTY_TYPE_0 => invalid [3] In practice, we can recognize binaries produced by old linkers when the object has been linked on a CET-enabled distribution because the produced notes are always invalid. The glibc dynamic loader already checks for this and does not enable CET in this case. ld -r involving exactly one CET-enabled object and one or more non-CET objects is still problematic, of course, but it seems an unlikely outcome. The only way I can see this happening is with a CET-by-default GCC (such as the one Ubuntu uses), but then you still would have to use another linker (not /usr/bin/ld). So even that seems like a fringe issue to me. In short, we looked at this situation, and still think that it's supportable. > Also note that sh_addralign(.note.gnu.property)=8 on a 64-bit > platform, while > sh_addralign(.note.gnu.build-id)=sh_addralign(.note.ABI-tag)=...=4 > (ancient mistake made by at least Linux/FreeBSD/NetBSD/...) GNU ld > before PR ld/23658 may create corrupted PT_NOTE. Yes, we ran into some of these issues in our distribution and had to rebuild a few objects. > For at least the above reasons, loaders are better not interpreting PT_NOTE. > glibc/sysdeps/x86/dl-prop.h is currently interpreting PT_NOTE => it > should be fixed. I'm not sure we can do that for backwards compatibility reasons. The ABI has been out there for several years now. > Given point 1 and 3, this comment deserves a reconsideration: > > > Binaries with .note.gnu.property section have been put into many > > OS releases. We must support them. > > 2. .note.gnu.property behaves strangely, unlike a regular SHT_NOTE. > For a .note.gnu.property aware linker (newer GNU ld, newer lld), > .note.gnu.property input sections are dropped. > > (We have .note.GNU-stack and .note.GNU-split-stack which both require special processing, but > they are SHT_PROGBITS.) > > 3. We need SHT_GNU_PROPERTY. > The output .note.gnu.property being SHT_NOTE causes linkers to place the section in both PT_NOTE > and PT_GNU_PROPERTY. > PT_NOTE, as explained by point 1 above, can cause trouble to old loaders. > Have we proved that "older linker-produced concatenated PT_NOTE cannot cause trouble to loaders interpreting PT_NOTE"? > > SHT_GNU_PROPERTY does not contribute to PT_NOTE and will not cause any problem to old loaders > interpreting PT_NOTE. Yes, I agree that it's desirable to add SHT_GNU_PROPERTY. Thanks, Florian ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-04-01 8:46 ` Florian Weimer @ 2020-04-01 9:22 ` Szabolcs Nagy 2020-04-01 9:29 ` Florian Weimer 0 siblings, 1 reply; 27+ messages in thread From: Szabolcs Nagy @ 2020-04-01 9:22 UTC (permalink / raw) To: Florian Weimer Cc: Fangrui Song, GNU C Library, gnu-gabi, Mark Wielaard, Zhang, Annita, Binutils, Cary Coutant, Liu, Hongtao The 04/01/2020 10:46, Florian Weimer via Libc-alpha wrote: > * Fangrui Song: > > > Below is my understanding of these matters. Hope they will be useful for > > interested stakeholders (for example, AArch64 devs, though PT_GNU_PROPERTY is > > currently driven by x86) who don't follow the discussions so closely. > > > > 1. We need PT_GNU_PROPERTY. > > > Old linkers don't know the special processing on input .note.gnu.property sections. > > The output .note.gnu.property does not take -z ibt/-z shstk/-z force-bti/-z pac-plt into account => > > invalid. > > The produced PT_NOTE may contain multiple NT_GNU_PROPERTY_TYPE_0 => invalid [3] > > In practice, we can recognize binaries produced by old linkers when the > object has been linked on a CET-enabled distribution because the > produced notes are always invalid. The glibc dynamic loader already > checks for this and does not enable CET in this case. > > ld -r involving exactly one CET-enabled object and one or more non-CET > objects is still problematic, of course, but it seems an unlikely > outcome. The only way I can see this happening is with a CET-by-default > GCC (such as the one Ubuntu uses), but then you still would have to use > another linker (not /usr/bin/ld). So even that seems like a fringe > issue to me. why only -r is problematic? i thought linking exactly one marked object and other non-marked ones with an old linker will have the (incorrect) marking on the output that cannot be recognised as wrong. this is why my plan for aarch64 is to only check PT_GNU_PROPERTY in glibc (which implies a new linker), i believe the kernel patches do so too. > > In short, we looked at this situation, and still think that it's > supportable. > > > Also note that sh_addralign(.note.gnu.property)=8 on a 64-bit > > platform, while > > sh_addralign(.note.gnu.build-id)=sh_addralign(.note.ABI-tag)=...=4 > > (ancient mistake made by at least Linux/FreeBSD/NetBSD/...) GNU ld > > before PR ld/23658 may create corrupted PT_NOTE. > > Yes, we ran into some of these issues in our distribution and had to > rebuild a few objects. > > > For at least the above reasons, loaders are better not interpreting PT_NOTE. > > glibc/sysdeps/x86/dl-prop.h is currently interpreting PT_NOTE => it > > should be fixed. > > I'm not sure we can do that for backwards compatibility reasons. The > ABI has been out there for several years now. on non-x86 targets it can be still done. > > Given point 1 and 3, this comment deserves a reconsideration: > > > > > Binaries with .note.gnu.property section have been put into many > > > OS releases. We must support them. > > > > 2. .note.gnu.property behaves strangely, unlike a regular SHT_NOTE. > > For a .note.gnu.property aware linker (newer GNU ld, newer lld), > > .note.gnu.property input sections are dropped. > > > > (We have .note.GNU-stack and .note.GNU-split-stack which both require special processing, but > > they are SHT_PROGBITS.) > > > > 3. We need SHT_GNU_PROPERTY. > > The output .note.gnu.property being SHT_NOTE causes linkers to place the section in both PT_NOTE > > and PT_GNU_PROPERTY. > > PT_NOTE, as explained by point 1 above, can cause trouble to old loaders. > > Have we proved that "older linker-produced concatenated PT_NOTE cannot cause trouble to loaders interpreting PT_NOTE"? > > > > SHT_GNU_PROPERTY does not contribute to PT_NOTE and will not cause any problem to old loaders > > interpreting PT_NOTE. > > Yes, I agree that it's desirable to add SHT_GNU_PROPERTY. +1 ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-04-01 9:22 ` Szabolcs Nagy @ 2020-04-01 9:29 ` Florian Weimer 2020-04-01 10:10 ` Szabolcs Nagy 0 siblings, 1 reply; 27+ messages in thread From: Florian Weimer @ 2020-04-01 9:29 UTC (permalink / raw) To: Szabolcs Nagy Cc: Fangrui Song, GNU C Library, gnu-gabi, Mark Wielaard, Zhang, Annita, Binutils, Cary Coutant, Liu, Hongtao * Szabolcs Nagy: > why only -r is problematic? > > i thought linking exactly one marked object and other non-marked > ones with an old linker will have the (incorrect) marking on the > output that cannot be recognised as wrong. Where do you get that single marked object? If you are on a CET-enabled distribution, the startup files are marked, so you have multiple marked objects right there. >> I'm not sure we can do that for backwards compatibility reasons. The >> ABI has been out there for several years now. > > on non-x86 targets it can be still done. That's true. Thanks, Florian ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-04-01 9:29 ` Florian Weimer @ 2020-04-01 10:10 ` Szabolcs Nagy 2020-04-01 10:21 ` Florian Weimer 0 siblings, 1 reply; 27+ messages in thread From: Szabolcs Nagy @ 2020-04-01 10:10 UTC (permalink / raw) To: Florian Weimer Cc: Fangrui Song, GNU C Library, gnu-gabi, Mark Wielaard, Zhang, Annita, Binutils, Cary Coutant, Liu, Hongtao The 04/01/2020 11:29, Florian Weimer wrote: > * Szabolcs Nagy: > > why only -r is problematic? > > > > i thought linking exactly one marked object and other non-marked > > ones with an old linker will have the (incorrect) marking on the > > output that cannot be recognised as wrong. > > Where do you get that single marked object? > > If you are on a CET-enabled distribution, the startup files are marked, > so you have multiple marked objects right there. assume your toolchain is not cet enabled (and has an old linker) but you link in a library archive that is cet enabled (because it was given to you that way) and exactly one .o from it gets used during the link. the resulting binary will have the marking and if deployed on a cet enabled system it won't work. (at least this is my current understanding.) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-04-01 10:10 ` Szabolcs Nagy @ 2020-04-01 10:21 ` Florian Weimer 0 siblings, 0 replies; 27+ messages in thread From: Florian Weimer @ 2020-04-01 10:21 UTC (permalink / raw) To: Szabolcs Nagy Cc: Fangrui Song, GNU C Library, gnu-gabi, Mark Wielaard, Zhang, Annita, Binutils, Cary Coutant, Liu, Hongtao * Szabolcs Nagy: > The 04/01/2020 11:29, Florian Weimer wrote: >> * Szabolcs Nagy: >> > why only -r is problematic? >> > >> > i thought linking exactly one marked object and other non-marked >> > ones with an old linker will have the (incorrect) marking on the >> > output that cannot be recognised as wrong. >> >> Where do you get that single marked object? >> >> If you are on a CET-enabled distribution, the startup files are marked, >> so you have multiple marked objects right there. > > assume your toolchain is not cet enabled (and has an old > linker) but you link in a library archive that is cet > enabled (because it was given to you that way) and exactly > one .o from it gets used during the link. > > the resulting binary will have the marking and if deployed > on a cet enabled system it won't work. (at least this is > my current understanding.) But that's not something we currently support. We do not have ABI stability for static linking like this. You are more likely to run into problems due to other issues (e.g., missing glibc symbols) than incorrect CET markup. Thanks, Florian ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` Fangrui Song 2020-01-01 0:00 ` H.J. Lu 1 sibling, 1 reply; 27+ messages in thread From: Fangrui Song @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu Cc: Mark Wielaard, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On 2020-02-19, H.J. Lu wrote: >On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <i@maskray.me> wrote: >> >> On 2020-02-19, H.J. Lu wrote: >> >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <mark@klomp.org> wrote: >> >> >> >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: >> >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: >> >> > > > https://patchwork.kernel.org/patch/11285409/ >> >> > > > >> >> > > > It is for both x86 and arm64. >> >> > > >> >> > > So that is not upstream in the mainline kernel? Why can't that patch >> >> > > use the existing PT_NOTE segment? That would make it compatible with >> >> > > existing binaries that don't have this PT_GNU_PROPERTY program header. >> >> > >> >> > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader >> >> > only wants to check PT_XXX. >> >> >> >> So they can check PT_NOTE because it provides the same information and >> >> is already available in existing binaries. >> >> >> > >> >Please take a look at glibc note.gnu.property parser. It is very complicated to >> >check for invalid .note.gnu.property sections generated by the older >> >linkers with >> >the new object. Kernel loader doesn't want to do it. >> >> One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html >> >> * Define SHT_GNU_PROPERTY >> * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY >> * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment >> >> The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). >> New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. >> ( https://patchwork.kernel.org/patch/11285409/ needs no change) >> >> This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. >> >> I will be happy to make changes to lld and LLVM binary utilities if this >> scheme reaches consensus. > >It is kind of too late now. Better late than never. It is never late to fix the section type if we do intend to fix it. Loaders don't read sections => the section type change is backward compatible. On 2020-02-20, H.J. Lu wrote: >On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <mark@klomp.org> wrote: >> >> Hi, >> >> On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote: >> > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <mark@klomp.org> wrote: >> > > This code isn't in the kernel yet. So either it gets changed to use the >> > > existing scheme with gnu property notes found through PT_NOTE to work >> > > with existing binaries. Then there is no need for PT_GNU_PROPERTY >> > > headers. >> > > >> > > Or some future kernel will start using PT_GNU_PROPERTY headers to find >> > > the gnu property notes. But that means it won't work with existing >> > > binaries that do not have that header. So there is no backwards >> > > compatibility anyway and we can define SHT_GNU_PROPERTY like above. >> > > >> > > So this actually seems the perfect time to make this decision. >> > >> > Binaries with .note.gnu.property section have been put into many >> > OS releases. We must support them. We can teach newer assemblers to emit SHT_GNU_PROPERTY. Newer linkers can support both SHT_GNU_PROPERTY/SHT_NOTE .note.gnu.property At some point in the future, linkers can drop support for SHT_NOTE .note.gnu.property Then it will become a graceful degradation: the old SHT_NOTE object files will not be different from older object files without .note.gnu.property >> OK. Then it is option 1. The kernel will need to support PT_NOTE for >> parsing the properties, since such older binaries won't have a >> PT_GNU_PROPERTY program header. Then we can simply get rid of >> PT_GNU_PROPERTY since nobody uses it and all information is already >> available through the PT_NOTE segment. >> > >Kernel loader only checks ld.so and static executable. Re-link them with >newer linker will get PT_GNU_PROPERTY. But ld.so needs to check >PT_NOTE for older binaries. The current PT_GNU_PROPERTY usage is all about hints. They are "nice to have" but not "necessary to have". I don't see any problem teaching newer loaders to forget PT_NOTE, if we do think PT_GNU_PROPERTY is the way forward. The currently mixed status is annoying: glibc: PT_NOTE Proposed Linux kernel patch: PT_GNU_PROPERTY ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Fangrui Song @ 2020-01-01 0:00 ` H.J. Lu 0 siblings, 0 replies; 27+ messages in thread From: H.J. Lu @ 2020-01-01 0:00 UTC (permalink / raw) To: Fangrui Song Cc: Mark Wielaard, Cary Coutant, Zhang, Annita, Liu, Hongtao, gnu-gabi, Binutils On Thu, Feb 20, 2020 at 1:43 PM Fangrui Song <i@maskray.me> wrote: > > On 2020-02-19, H.J. Lu wrote: > >On Wed, Feb 19, 2020 at 10:27 AM Fangrui Song <i@maskray.me> wrote: > >> > >> On 2020-02-19, H.J. Lu wrote: > >> >On Wed, Feb 19, 2020 at 5:17 AM Mark Wielaard <mark@klomp.org> wrote: > >> >> > >> >> On Wed, 2020-02-19 at 04:28 -0800, H.J. Lu wrote: > >> >> > On Wed, Feb 19, 2020 at 4:02 AM Mark Wielaard <mark@klomp.org> wrote: > >> >> > > > https://patchwork.kernel.org/patch/11285409/ > >> >> > > > > >> >> > > > It is for both x86 and arm64. > >> >> > > > >> >> > > So that is not upstream in the mainline kernel? Why can't that patch > >> >> > > use the existing PT_NOTE segment? That would make it compatible with > >> >> > > existing binaries that don't have this PT_GNU_PROPERTY program header. > >> >> > > >> >> > Kernel loader is one of motivations of PT_GNU_PROPERTY. Kernel loader > >> >> > only wants to check PT_XXX. > >> >> > >> >> So they can check PT_NOTE because it provides the same information and > >> >> is already available in existing binaries. > >> >> > >> > > >> >Please take a look at glibc note.gnu.property parser. It is very complicated to > >> >check for invalid .note.gnu.property sections generated by the older > >> >linkers with > >> >the new object. Kernel loader doesn't want to do it. > >> > >> One way to make things follow the spirit of https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html > >> > >> * Define SHT_GNU_PROPERTY > >> * Set sh_type(.note.gnu.property) to SHT_GNU_PROPERTY > >> * Place SHT_GNU_PROPERTY sections in a PT_GNU_PROPERTY segment > >> > >> The generated PT_NOTE will not include .note.gnu.property, so the scheme is compatible with old loaders (ld.so, gdb, Linux, etc). > >> New loaders should interpret PT_GNU_PROPERTY, instead of PT_NOTE. > >> ( https://patchwork.kernel.org/patch/11285409/ needs no change) > >> > >> This way linkers can keep treating SHT_NOTE sections as opaque and apply "Rules for Linking Unrecognized Sections" (http://www.sco.com/developers/gabi/latest/ch4.sheader.html ) when combining SHT_NOTE sections. At least for lld, there will be no special rules for input SHT_NOTE sections. > >> > >> I will be happy to make changes to lld and LLVM binary utilities if this > >> scheme reaches consensus. > > > >It is kind of too late now. > > Better late than never. It is never late to fix the section type if we do intend to fix it. > > Loaders don't read sections => the section type change is backward compatible. > > On 2020-02-20, H.J. Lu wrote: > >On Thu, Feb 20, 2020 at 1:37 AM Mark Wielaard <mark@klomp.org> wrote: > >> > >> Hi, > >> > >> On Wed, 2020-02-19 at 14:17 -0800, H.J. Lu wrote: > >> > On Wed, Feb 19, 2020 at 1:46 PM Mark Wielaard <mark@klomp.org> wrote: > >> > > This code isn't in the kernel yet. So either it gets changed to use the > >> > > existing scheme with gnu property notes found through PT_NOTE to work > >> > > with existing binaries. Then there is no need for PT_GNU_PROPERTY > >> > > headers. > >> > > > >> > > Or some future kernel will start using PT_GNU_PROPERTY headers to find > >> > > the gnu property notes. But that means it won't work with existing > >> > > binaries that do not have that header. So there is no backwards > >> > > compatibility anyway and we can define SHT_GNU_PROPERTY like above. > >> > > > >> > > So this actually seems the perfect time to make this decision. > >> > > >> > Binaries with .note.gnu.property section have been put into many > >> > OS releases. We must support them. > > We can teach newer assemblers to emit SHT_GNU_PROPERTY. > Newer linkers can support both SHT_GNU_PROPERTY/SHT_NOTE .note.gnu.property > > At some point in the future, linkers can drop support for SHT_NOTE .note.gnu.property > Then it will become a graceful degradation: the old SHT_NOTE object files will not be > different from older object files without .note.gnu.property > > >> OK. Then it is option 1. The kernel will need to support PT_NOTE for > >> parsing the properties, since such older binaries won't have a > >> PT_GNU_PROPERTY program header. Then we can simply get rid of > >> PT_GNU_PROPERTY since nobody uses it and all information is already > >> available through the PT_NOTE segment. > >> > > > >Kernel loader only checks ld.so and static executable. Re-link them with > >newer linker will get PT_GNU_PROPERTY. But ld.so needs to check > >PT_NOTE for older binaries. > > The current PT_GNU_PROPERTY usage is all about hints. They are "nice to have" but not > "necessary to have". I don't see any problem teaching newer loaders to forget > PT_NOTE, if we do think PT_GNU_PROPERTY is the way forward. > > The currently mixed status is annoying: > > glibc: PT_NOTE > Proposed Linux kernel patch: PT_GNU_PROPERTY Since this has been deployed on Linux, any changes should be discussed at https://sourceware.org/ml/gnu-gabi/ -- H.J. ^ permalink raw reply [flat|nested] 27+ messages in thread
* RE: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` Fangrui Song via gnu-gabi 2020-01-01 0:00 ` Mark Wielaard @ 2020-01-01 0:00 ` Zhang, Annita 1 sibling, 0 replies; 27+ messages in thread From: Zhang, Annita @ 2020-01-01 0:00 UTC (permalink / raw) To: Fangrui Song, H.J. Lu; +Cc: Mark Wielaard, gnu-gabi, Binutils, Liu, Hongtao Regarding x86-64 ABI, we have internal communication if there's some update. And we will raise it up in LLVM community if necessary. Thanks, Annita -----Original Message----- From: Fangrui Song <maskray@google.com> Sent: Wednesday, February 19, 2020 10:31 AM To: H.J. Lu <hjl.tools@gmail.com>; Zhang, Annita <annita.zhang@intel.com> Cc: Mark Wielaard <mark@klomp.org>; gnu-gabi <gnu-gabi@sourceware.org>; Binutils <binutils@sourceware.org> Subject: Re: binutils ld and new PT_GNU_PROPERTY segment On 2020-02-18, H.J. Lu wrote: >On Tue, Feb 18, 2020@4:38 AM Mark Wielaard <mark@klomp.org> wrote: >> >> Hi, >> >> binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553) >> segment that overlaps with the PT_NOTE segment covering the >> .note.gnu.property section data. >> >> I cannot tell if this is an accident/experiment that happened to end >> up in a release or if it is proposed as an official new segment type. > >https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html >https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI > >> As far as I can tell binutils ld is the only linker which adds this > >Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not, it >is an lld bug. > >> extra segment and there is no runtime loader which uses it. It >> doesn't provide any new information that cannot be found in the >> existing PT_NOTE segment. > >Kernel loader uses it for both ARM and x86. > >> On 64 bit architectures it simply covers the extra existing PT_NOTE >> with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). >> On 32bit architectures it covers a sub-range of the existing PT_NOTE >> segment. >> >> It isn't clear to me how other tools should handle this. It seems to >> prevent normal merging of note sections. Since some notes are >> probably special if they need to be covered by this new segment type. >> And it isn't clear how the linker knows which of the SHT_NOTE >> sections is what needs to be covered by the new segment type. Or is >> the idea that this will eventually come with a new section type too >> and GNU properties will no longer use NOTE sections? >> > >PT_GNU_PROPERTY covers .note.gnu.property section. https://reviews.llvm.org/D70961 added PT_GNU_PROPERTY support to lld. The change will be included by lld 10.0.0 (currently at rc2). https://reviews.llvm.org/D70959 added the dump support to llvm-readelf -l and llvm-objdump -p. The change will be included by LLVM 10.0.0 (currently at rc2). From what I can see, neither the Linux kernel nor glibc uses PT_GNU_PROPERTY. glibc/sysdeps/x86/dl-prop.h parses PT_NOTE. I tend to agree with Cary (https://sourceware.org/ml/gnu-gabi/2018-q4/msg00036.html) that .note.gnu.property should have been designed as a different section type because its combining semantics are different from other notes (we could apply "Rules for Linking Unrecognized Sections" to all SHT_NOTE sections) but it is too late to change the section type. A separate segment type (PT_GNU_PROPERTY) looks fine to me. glibc should probably be updated to parse PT_GNU_PROPERTY instead. (Recently I read some ABI decisions and I noticed that I frequently see the term "it is too late". As a contributor of both lld and LLVM binary utilities (and the implementer of a bunch of GNU_PROPERTY changes), I hope that the LLVM community can be informed of such changes earlier. A lot of people are not subscribed to any of the mailing lists (recently I visit the archives from time to time). Looks like Annita's job? :) ) ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: binutils ld and new PT_GNU_PROPERTY segment 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Fangrui Song via gnu-gabi @ 2020-01-01 0:00 ` Mark Wielaard 1 sibling, 0 replies; 27+ messages in thread From: Mark Wielaard @ 2020-01-01 0:00 UTC (permalink / raw) To: H.J. Lu; +Cc: Rahul Chaudhry via gnu-gabi, Binutils, Zhang, Annita Hi, On Tue, 2020-02-18 at 04:48 -0800, H.J. Lu wrote: > On Tue, Feb 18, 2020 at 4:38 AM Mark Wielaard <mark@klomp.org> wrote: > > binutils 2.32 ld emits a new PT_GNU_PROPERTY (PT_LOOS + 0x474e553) > > segment that overlaps with the PT_NOTE segment covering the > > .note.gnu.property section data. > > > > I cannot tell if this is an accident/experiment that happened to > > end up > > in a release or if it is proposed as an official new segment type. > > https://sourceware.org/ml/gnu-gabi/2018-q4/msg00027.html > https://github.com/hjl-tools/linux-abi/wiki/Linux-Extensions-to-gABI I see you added it, but I don't see consensus for it. It also doesn't really make sense if it still uses the same section type and has to rely on a magic section name. > > As far as I can tell binutils ld is the only linker which adds this > > Annita, does lld generate PT_GNU_PROPERTY segment with CET? If not, > it is an lld bug. I think it is the other way around, binutils ld generates it, but nobody uses it, it is redundant and the interaction with other tools dealing with SHT_NOTE sections isn't really clear. > > extra segment and there is no runtime loader which uses it. It > > doesn't > > provide any new information that cannot be found in the existing > > PT_NOTE segment. > > Kernel loader uses it for both ARM and x86. I cannot find it. Could you point to where it uses it? Why doesn't it simply use the PT_NOTE segment? > > On 64 bit architectures it simply covers the extra existing PT_NOTE > > with 8 byte alignment (normal PT_NOTE segments are 4 byte aligned). On > > 32bit architectures it covers a sub-range of the existing PT_NOTE > > segment. > > > > It isn't clear to me how other tools should handle this. It seems to > > prevent normal merging of note sections. Since some notes are probably > > special if they need to be covered by this new segment type. And it > > isn't clear how the linker knows which of the SHT_NOTE sections is what > > needs to be covered by the new segment type. Or is the idea that this > > will eventually come with a new section type too and GNU properties > > will no longer use NOTE sections? > > > > PT_GNU_PROPERTY covers .note.gnu.property section. Using magic section names is a problem. It means other tools can no longer rely on the section type. Can we please simply remove this because it isn't used, doesn't provide any new information, depends on magic section names and makes it harder for other tools to deal with generic SHT_NOTE sections. Thanks, Mark ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2020-04-01 10:21 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-01-01 0:00 binutils ld and new PT_GNU_PROPERTY segment Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Fangrui Song via gnu-gabi 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Fangrui Song 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Mark Wielaard 2020-01-01 0:00 ` Fangrui Song 2020-04-01 8:46 ` Florian Weimer 2020-04-01 9:22 ` Szabolcs Nagy 2020-04-01 9:29 ` Florian Weimer 2020-04-01 10:10 ` Szabolcs Nagy 2020-04-01 10:21 ` Florian Weimer 2020-01-01 0:00 ` Fangrui Song 2020-01-01 0:00 ` H.J. Lu 2020-01-01 0:00 ` Zhang, Annita 2020-01-01 0:00 ` Mark Wielaard
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).