* Add -fdirect-access-external-data @ 2020-12-17 7:10 Fangrui Song 2020-12-26 15:31 ` Florian Weimer 0 siblings, 1 reply; 7+ messages in thread From: Fangrui Song @ 2020-12-17 7:10 UTC (permalink / raw) To: gcc Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which proposes -fdirect-access-external-data to address some x86-64 GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] it can prevent copy relocations). [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 Since I am going to add this option to Clang and I hope (once GCC decides to implement this option the two compilers can use the same option name), I bring it to your attention. Please CC me on replies:) ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Add -fdirect-access-external-data 2020-12-17 7:10 Add -fdirect-access-external-data Fangrui Song @ 2020-12-26 15:31 ` Florian Weimer 2020-12-26 15:39 ` H.J. Lu 0 siblings, 1 reply; 7+ messages in thread From: Florian Weimer @ 2020-12-26 15:31 UTC (permalink / raw) To: Fangrui Song; +Cc: gcc * Fangrui Song: > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > proposes -fdirect-access-external-data to address some x86-64 > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > it can prevent copy relocations). > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > Since I am going to add this option to Clang and I hope (once GCC decides to > implement this option the two compilers can use the same option name), I bring > it to your attention. One worry I have is that people start building shared objects with direct data access, expecting the main program to be built with indirect access. We already see this today with Qt. It's not really supported well by the toolchain and causes frequent issues. Depending on the ELF ABI in question, the new pair of -f options might not actually be meaningful. It really depends on whether you have reasonably-sized displacements available. I think there are some ABIs where the optimization is theoretically possible, but impractical because the ilimit it imposes on data segment (think AArch64 without adrp). ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Add -fdirect-access-external-data 2020-12-26 15:31 ` Florian Weimer @ 2020-12-26 15:39 ` H.J. Lu 2021-01-07 6:26 ` Fangrui Song [not found] ` <MWHPR1201MB011025EE98F5F40BA9AE62E0CBAF0@MWHPR1201MB0110.namprd12.prod.outlook.com> 0 siblings, 2 replies; 7+ messages in thread From: H.J. Lu @ 2020-12-26 15:39 UTC (permalink / raw) To: Florian Weimer; +Cc: Fangrui Song, GCC Development On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > * Fangrui Song: > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > proposes -fdirect-access-external-data to address some x86-64 > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > it can prevent copy relocations). > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > implement this option the two compilers can use the same option name), I bring > > it to your attention. > > One worry I have is that people start building shared objects with > direct data access, expecting the main program to be built with > indirect access. We already see this today with Qt. It's not really > supported well by the toolchain and causes frequent issues. It can be solved by ABI extension implemented in linker, ld.so and compiler. > Depending on the ELF ABI in question, the new pair of -f options might > not actually be meaningful. It really depends on whether you have > reasonably-sized displacements available. I think there are some ABIs > where the optimization is theoretically possible, but impractical > because the ilimit it imposes on data segment (think AArch64 without > adrp). -- H.J. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Add -fdirect-access-external-data 2020-12-26 15:39 ` H.J. Lu @ 2021-01-07 6:26 ` Fangrui Song [not found] ` <MWHPR1201MB011025EE98F5F40BA9AE62E0CBAF0@MWHPR1201MB0110.namprd12.prod.outlook.com> 1 sibling, 0 replies; 7+ messages in thread From: Fangrui Song @ 2021-01-07 6:26 UTC (permalink / raw) To: GCC Development; +Cc: H.J. Lu, Florian Weimer On Sat, Dec 26, 2020 at 7:39 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > > > * Fangrui Song: > > > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > > proposes -fdirect-access-external-data to address some x86-64 > > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > > it can prevent copy relocations). > > > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > > implement this option the two compilers can use the same option name), I bring > > > it to your attention. > > > > One worry I have is that people start building shared objects with > > direct data access, expecting the main program to be built with > > indirect access. We already see this today with Qt. It's not really > > supported well by the toolchain and causes frequent issues. > > It can be solved by ABI extension implemented in linker, ld.so and > compiler. > > > Depending on the ELF ABI in question, the new pair of -f options might > > not actually be meaningful. It really depends on whether you have > > reasonably-sized displacements available. I think there are some ABIs > > where the optimization is theoretically possible, but impractical > > because the ilimit it imposes on data segment (think AArch64 without > > adrp). > > > > -- > H.J. Please check out new comments on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 -fdirect-access-external-data is still the best name. The option is useful to avoid copy relocations / "canonical PLT entry" (st_shndx=0,st_value!=0) in -fno-pic code. I will proceed with my Clang patch. ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <MWHPR1201MB011025EE98F5F40BA9AE62E0CBAF0@MWHPR1201MB0110.namprd12.prod.outlook.com>]
* Re: Add -fdirect-access-external-data [not found] ` <MWHPR1201MB011025EE98F5F40BA9AE62E0CBAF0@MWHPR1201MB0110.namprd12.prod.outlook.com> @ 2021-01-08 2:07 ` H.J. Lu 2021-01-08 3:38 ` Fangrui Song [not found] ` <MWHPR1201MB01103E59742F469B75B35BDBCBAE0@MWHPR1201MB0110.namprd12.prod.outlook.com> 0 siblings, 2 replies; 7+ messages in thread From: H.J. Lu @ 2021-01-08 2:07 UTC (permalink / raw) To: Fangrui Song; +Cc: GCC Development, Florian Weimer On Wed, Jan 6, 2021 at 10:32 PM Fangrui Song <i@maskray.me> wrote: > > On Sat, Dec 26, 2020 at 7:39 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > > > > > * Fangrui Song: > > > > > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > > > proposes -fdirect-access-external-data to address some x86-64 > > > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > > > it can prevent copy relocations). > > > > > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > > > implement this option the two compilers can use the same option name), I bring > > > > it to your attention. > > > > > > One worry I have is that people start building shared objects with > > > direct data access, expecting the main program to be built with > > > indirect access. We already see this today with Qt. It's not really > > > supported well by the toolchain and causes frequent issues. > > > > It can be solved by ABI extension implemented in linker, ld.so and > > compiler. > > > > > Depending on the ELF ABI in question, the new pair of -f options might > > > not actually be meaningful. It really depends on whether you have > > > reasonably-sized displacements available. I think there are some ABIs > > > where the optimization is theoretically possible, but impractical > > > because the ilimit it imposes on data segment (think AArch64 without > > > adrp). > > > > > > > > -- > > H.J. > > Please check out new comments on > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 > > -fdirect-access-external-data is still the best name. The option is > useful to avoid copy relocations / "canonical PLT entry" > (st_shndx=0,st_value!=0) in -fno-pic code. > I will proceed with my Clang patch. If I understand it correctly, you want to treat all accesses to protected definitions as local access and all read/write accesses to undefined symbols should go through GOT. Branches to undefined symbols can use PLT. -fdirect-access-external-data doesn't reflect it. -- H.J. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Add -fdirect-access-external-data 2021-01-08 2:07 ` H.J. Lu @ 2021-01-08 3:38 ` Fangrui Song [not found] ` <MWHPR1201MB01103E59742F469B75B35BDBCBAE0@MWHPR1201MB0110.namprd12.prod.outlook.com> 1 sibling, 0 replies; 7+ messages in thread From: Fangrui Song @ 2021-01-08 3:38 UTC (permalink / raw) To: H.J. Lu; +Cc: GCC Development, Florian Weimer On Thu, Jan 7, 2021 at 6:07 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > On Wed, Jan 6, 2021 at 10:32 PM Fangrui Song <i@maskray.me> wrote: > > > > On Sat, Dec 26, 2020 at 7:39 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > > > > > > > * Fangrui Song: > > > > > > > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > > > > proposes -fdirect-access-external-data to address some x86-64 > > > > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > > > > it can prevent copy relocations). > > > > > > > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > > > > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > > > > implement this option the two compilers can use the same option name), I bring > > > > > it to your attention. > > > > > > > > One worry I have is that people start building shared objects with > > > > direct data access, expecting the main program to be built with > > > > indirect access. We already see this today with Qt. It's not really > > > > supported well by the toolchain and causes frequent issues. > > > > > > It can be solved by ABI extension implemented in linker, ld.so and > > > compiler. > > > > > > > Depending on the ELF ABI in question, the new pair of -f options might > > > > not actually be meaningful. It really depends on whether you have > > > > reasonably-sized displacements available. I think there are some ABIs > > > > where the optimization is theoretically possible, but impractical > > > > because the ilimit it imposes on data segment (think AArch64 without > > > > adrp). > > > > > > > > > > > > -- > > > H.J. > > > > Please check out new comments on > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 > > > > -fdirect-access-external-data is still the best name. The option is > > useful to avoid copy relocations / "canonical PLT entry" > > (st_shndx=0,st_value!=0) in -fno-pic code. > > I will proceed with my Clang patch. > > If I understand it correctly, you want to treat all accesses to protected > definitions as local access > and all read/write accesses to undefined symbols > should go through GOT. Branches to undefined symbols can use PLT. > -fdirect-access-external-data doesn't reflect it. My apologies. Direct/indirect access to protected definitions is a separate topic, unrelated to -f[no-]direct-access-external-data. ( If anyone is interested, there was a heated discussion about accesses to protected definitions https://sourceware.org/legacy-ml/binutils/2016-03/msg00312.html basically a lot of folks considered that copy relocations are best-effort support provided by the toolchain. For protected symbols, copy relocations do not necessarily work. Clang always treats protected similar to hidden/internal, no special logic for x86-64 protected. ) Branches to undefined symbols is yet another separate topic. ( On x86-64, there is no PIC vs non-PIC PLT distinction and an R_X86_64_PLT32 relocation is produced for both `call/jmp foo` and `call/jmp foo@PLT` with newer (2018) GNU as/LLVM integrated assembler. On i386, there are 2 types of PLTs, PIC and non-PIC. Currently the informal convention is to use R_386_PC32 for non-PIC PLT and R_386_PLT32 for PIC PLT, but R_386_PLT32 is arguably preferable for -fno-pic code as well: this can avoid a "canonical PLT entry" (st_shndx=0, st_value!=0) if the symbol turns out to be defined externally. My idea is that we can always use R_386_PLT32 in -fno-pic mode. ) Taking the address of an external function is related to -f[no-]direct-access-external-data. A function pointer of an external function is very similar to external data. A canonical PLT entry can be caused by either a branch (R_386_PC32/R_386_32) or an address taken operation (R_386_PC32/R_386_32) if the symbol turns out to be external. -fno-direct-access-external-data can only address the function pointer usage. ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <MWHPR1201MB01103E59742F469B75B35BDBCBAE0@MWHPR1201MB0110.namprd12.prod.outlook.com>]
* Re: Add -fdirect-access-external-data [not found] ` <MWHPR1201MB01103E59742F469B75B35BDBCBAE0@MWHPR1201MB0110.namprd12.prod.outlook.com> @ 2021-01-08 4:11 ` H.J. Lu 0 siblings, 0 replies; 7+ messages in thread From: H.J. Lu @ 2021-01-08 4:11 UTC (permalink / raw) To: Fangrui Song; +Cc: GCC Development, Florian Weimer On Thu, Jan 7, 2021 at 7:45 PM Fangrui Song <i@maskray.me> wrote: > > On Thu, Jan 7, 2021 at 6:07 PM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > On Wed, Jan 6, 2021 at 10:32 PM Fangrui Song <i@maskray.me> wrote: > > > > > > On Sat, Dec 26, 2020 at 7:39 AM H.J. Lu <hjl.tools@gmail.com> wrote: > > > > > > > > On Sat, Dec 26, 2020 at 7:32 AM Florian Weimer <fw@deneb.enyo.de> wrote: > > > > > > > > > > * Fangrui Song: > > > > > > > > > > > Hi, I filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 which > > > > > > proposes -fdirect-access-external-data to address some x86-64 > > > > > > GCC/binutils pain[1] and also benefit non-x86 architectures (also see [1] > > > > > > it can prevent copy relocations). > > > > > > > > > > > > [1] Mentioned in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112#c2 > > > > > > > > > > > > Since I am going to add this option to Clang and I hope (once GCC decides to > > > > > > implement this option the two compilers can use the same option name), I bring > > > > > > it to your attention. > > > > > > > > > > One worry I have is that people start building shared objects with > > > > > direct data access, expecting the main program to be built with > > > > > indirect access. We already see this today with Qt. It's not really > > > > > supported well by the toolchain and causes frequent issues. > > > > > > > > It can be solved by ABI extension implemented in linker, ld.so and > > > > compiler. > > > > > > > > > Depending on the ELF ABI in question, the new pair of -f options might > > > > > not actually be meaningful. It really depends on whether you have > > > > > reasonably-sized displacements available. I think there are some ABIs > > > > > where the optimization is theoretically possible, but impractical > > > > > because the ilimit it imposes on data segment (think AArch64 without > > > > > adrp). > > > > > > > > > > > > > > > > -- > > > > H.J. > > > > > > Please check out new comments on > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98112 > > > > > > -fdirect-access-external-data is still the best name. The option is > > > useful to avoid copy relocations / "canonical PLT entry" > > > (st_shndx=0,st_value!=0) in -fno-pic code. > > > I will proceed with my Clang patch. > > > > If I understand it correctly, you want to treat all accesses to protected > > definitions as local access > > and all read/write accesses to undefined symbols > > should go through GOT. Branches to undefined symbols can use PLT. > > -fdirect-access-external-data doesn't reflect it. > > My apologies. Direct/indirect access to protected definitions is a separate > topic, unrelated to -f[no-]direct-access-external-data. > > ( > If anyone is interested, there was a heated discussion about accesses to > protected definitions > https://sourceware.org/legacy-ml/binutils/2016-03/msg00312.html basically a lot > of folks considered that copy relocations are best-effort support provided by > the toolchain. For protected symbols, copy relocations do not necessarily work. > > Clang always treats protected similar to hidden/internal, no special > logic for x86-64 protected. > ) Then, function pointer and copy relocation don't work correctly on protected symbols with clang. For GCC, function pointer and copy relocation on protected symbols work correctly today. > Branches to undefined symbols is yet another separate topic. > All these issues are related. > ( > On x86-64, there is no PIC vs non-PIC PLT distinction and an R_X86_64_PLT32 > relocation is produced for both `call/jmp foo` and `call/jmp foo@PLT` with newer > (2018) GNU as/LLVM integrated assembler. > > On i386, there are 2 types of PLTs, PIC and non-PIC. Currently the informal > convention is to use R_386_PC32 for non-PIC PLT and R_386_PLT32 for PIC PLT, but > R_386_PLT32 is arguably preferable for -fno-pic code as well: this can avoid a > "canonical PLT entry" (st_shndx=0, st_value!=0) if the symbol turns out to be > defined externally. > > My idea is that we can always use R_386_PLT32 in -fno-pic mode. > ) > > Taking the address of an external function is related to > -f[no-]direct-access-external-data. A function pointer of an external function > is very similar to external data. > > A canonical PLT entry can be caused by either a branch (R_386_PC32/R_386_32) > or an address taken operation (R_386_PC32/R_386_32) if the symbol > turns out to be external. > -fno-direct-access-external-data can only address the function pointer usage. i386 is legacy. We should leave it alone. But we should do something for x86-64. That is why I proposed an ABI change: https://groups.google.com/g/x86-64-abi/c/DRvKxJ1AH3Q What I'd like to see is a compiler option which does 1. In shared object, all accesses to protected definitions can be treated as local access. 2. In PIE, all read/write accesses to undefined symbols should go through GOT. 3. In PIE and shared object, all global function pointers, whose function bodies aren't defined locally, should be resolved to GOT. 4. Branches to undefined symbols use PLT. -f[no-]direct-access-external-data doesn't properly describe these. Maybe -fone-global-definition is better. -- H.J. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-08 4:12 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-17 7:10 Add -fdirect-access-external-data Fangrui Song 2020-12-26 15:31 ` Florian Weimer 2020-12-26 15:39 ` H.J. Lu 2021-01-07 6:26 ` Fangrui Song [not found] ` <MWHPR1201MB011025EE98F5F40BA9AE62E0CBAF0@MWHPR1201MB0110.namprd12.prod.outlook.com> 2021-01-08 2:07 ` H.J. Lu 2021-01-08 3:38 ` Fangrui Song [not found] ` <MWHPR1201MB01103E59742F469B75B35BDBCBAE0@MWHPR1201MB0110.namprd12.prod.outlook.com> 2021-01-08 4:11 ` H.J. Lu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).