[RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Richard Henderson]

[ I never saw this arrive in the list archives.
  Trying again with my linaro address subscribed. ]

We should default to the larger code model, in order to support
larger applications built with -static -pie.  This should be
consistent with pic-ccflag, which defaults to -fPIC.

Remove the now redundant override from sysdeps/sparc/Makefile.
Note that -fno-pie and -fno-PIE have the same effect.

---

In the case of AArch64, the code changes are small but significant:

 0000000000000000 <__libc_init_first>:
    0:  a9bd7bfd        stp     x29, x30, [sp, #-48]!
-   4:  90000003        adrp    x3, 0 <_GLOBAL_OFFSET_TABLE_>
-                       4: R_AARCH64_ADR_PREL_PG_HI21   _GLOBAL_OFFSET_TABLE_
+   4:  90000003        adrp    x3, 0 <__environ>
+                       4: R_AARCH64_ADR_GOT_PAGE       __environ
    8:  90000004        adrp    x4, 0 <__libc_init_first>
                        8: R_AARCH64_ADR_PREL_PG_HI21   .bss
    c:  910003fd        mov     x29, sp
   10:  f9400063        ldr     x3, [x3]
-                       10: R_AARCH64_LD64_GOTPAGE_LO15 __environ
+                       10: R_AARCH64_LD64_GOT_LO12_NC  __environ

In the small model, we are constrained to 15 bits of GOT, with a
single shared base, _GLOBAL_OFFSET_TABLE_.  In the large model,
each symbol has a page + offset pair.  For small functions like
this, where there are no other variable references to share the
common GOT pointer, there is no actual code change.

r~
---
 Makeconfig             | 2 +-
 sysdeps/sparc/Makefile | 3 ---
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/Makeconfig b/Makeconfig
index e78cf220af..f8164a0025 100644
--- a/Makeconfig
+++ b/Makeconfig
@@ -1052,7 +1052,7 @@ pic-ccflag = -fPIC
 PIC-ccflag = -fPIC
 endif
 # This can be changed by a sysdep makefile
-pie-ccflag = -fpie
+pie-ccflag = -fPIE
 no-pie-ccflag = -fno-pie
 # This one should always stay like this unless there is a very good reason.
 PIE-ccflag = -fPIE
diff --git a/sysdeps/sparc/Makefile b/sysdeps/sparc/Makefile
index 12c2c1b085..26b4a84606 100644
--- a/sysdeps/sparc/Makefile
+++ b/sysdeps/sparc/Makefile
@@ -1,9 +1,6 @@
 # The Sparc `long double' is a distinct type we support.
 long-double-fcts = yes
 
-pie-ccflag = -fPIE
-no-pie-ccflag = -fno-PIE
-
 ifeq ($(subdir),gmon)
 sysdep_routines += sparc-mcount
 endif
-- 
2.34.1

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Adhemerval Zanella Netto]

On 25/08/22 18:01, Richard Henderson via Libc-alpha wrote:
> [ I never saw this arrive in the list archives.
>   Trying again with my linaro address subscribed. ]
> 
> We should default to the larger code model, in order to support
> larger applications built with -static -pie.  This should be
> consistent with pic-ccflag, which defaults to -fPIC.
> 
> Remove the now redundant override from sysdeps/sparc/Makefile.
> Note that -fno-pie and -fno-PIE have the same effect.
> 
> ---
> 
> In the case of AArch64, the code changes are small but significant:
> 
>  0000000000000000 <__libc_init_first>:
>     0:  a9bd7bfd        stp     x29, x30, [sp, #-48]!
> -   4:  90000003        adrp    x3, 0 <_GLOBAL_OFFSET_TABLE_>
> -                       4: R_AARCH64_ADR_PREL_PG_HI21   _GLOBAL_OFFSET_TABLE_
> +   4:  90000003        adrp    x3, 0 <__environ>
> +                       4: R_AARCH64_ADR_GOT_PAGE       __environ
>     8:  90000004        adrp    x4, 0 <__libc_init_first>
>                         8: R_AARCH64_ADR_PREL_PG_HI21   .bss
>     c:  910003fd        mov     x29, sp
>    10:  f9400063        ldr     x3, [x3]
> -                       10: R_AARCH64_LD64_GOTPAGE_LO15 __environ
> +                       10: R_AARCH64_LD64_GOT_LO12_NC  __environ
> 
> In the small model, we are constrained to 15 bits of GOT, with a
> single shared base, _GLOBAL_OFFSET_TABLE_.  In the large model,
> each symbol has a page + offset pair.  For small functions like
> this, where there are no other variable references to share the
> common GOT pointer, there is no actual code change.
> 
> r~

I recall that powerpc wanted to use -mcmodel=large, but it would require to
adjust a lot of assembly implementations.  Are we sure this change does not 
trigger any regression with current glibc code?

> ---
>  Makeconfig             | 2 +-
>  sysdeps/sparc/Makefile | 3 ---
>  2 files changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/Makeconfig b/Makeconfig
> index e78cf220af..f8164a0025 100644
> --- a/Makeconfig
> +++ b/Makeconfig
> @@ -1052,7 +1052,7 @@ pic-ccflag = -fPIC
>  PIC-ccflag = -fPIC
>  endif
>  # This can be changed by a sysdep makefile
> -pie-ccflag = -fpie
> +pie-ccflag = -fPIE
>  no-pie-ccflag = -fno-pie
>  # This one should always stay like this unless there is a very good reason.
>  PIE-ccflag = -fPIE
> diff --git a/sysdeps/sparc/Makefile b/sysdeps/sparc/Makefile
> index 12c2c1b085..26b4a84606 100644
> --- a/sysdeps/sparc/Makefile
> +++ b/sysdeps/sparc/Makefile
> @@ -1,9 +1,6 @@
>  # The Sparc `long double' is a distinct type we support.
>  long-double-fcts = yes
>  
> -pie-ccflag = -fPIE
> -no-pie-ccflag = -fno-PIE
> -
>  ifeq ($(subdir),gmon)
>  sysdep_routines += sparc-mcount
>  endif

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Richard Henderson]

On 8/26/22 05:32, Adhemerval Zanella Netto wrote:
> 
> 
> On 25/08/22 18:01, Richard Henderson via Libc-alpha wrote:
>> [ I never saw this arrive in the list archives.
>>    Trying again with my linaro address subscribed. ]
>>
>> We should default to the larger code model, in order to support
>> larger applications built with -static -pie.  This should be
>> consistent with pic-ccflag, which defaults to -fPIC.
>>
>> Remove the now redundant override from sysdeps/sparc/Makefile.
>> Note that -fno-pie and -fno-PIE have the same effect.
>>
>> ---
>>
>> In the case of AArch64, the code changes are small but significant:
>>
>>   0000000000000000 <__libc_init_first>:
>>      0:  a9bd7bfd        stp     x29, x30, [sp, #-48]!
>> -   4:  90000003        adrp    x3, 0 <_GLOBAL_OFFSET_TABLE_>
>> -                       4: R_AARCH64_ADR_PREL_PG_HI21   _GLOBAL_OFFSET_TABLE_
>> +   4:  90000003        adrp    x3, 0 <__environ>
>> +                       4: R_AARCH64_ADR_GOT_PAGE       __environ
>>      8:  90000004        adrp    x4, 0 <__libc_init_first>
>>                          8: R_AARCH64_ADR_PREL_PG_HI21   .bss
>>      c:  910003fd        mov     x29, sp
>>     10:  f9400063        ldr     x3, [x3]
>> -                       10: R_AARCH64_LD64_GOTPAGE_LO15 __environ
>> +                       10: R_AARCH64_LD64_GOT_LO12_NC  __environ
>>
>> In the small model, we are constrained to 15 bits of GOT, with a
>> single shared base, _GLOBAL_OFFSET_TABLE_.  In the large model,
>> each symbol has a page + offset pair.  For small functions like
>> this, where there are no other variable references to share the
>> common GOT pointer, there is no actual code change.
>>
>> r~
> 
> I recall that powerpc wanted to use -mcmodel=large, but it would require to
> adjust a lot of assembly implementations.  Are we sure this change does not
> trigger any regression with current glibc code?

Ah, the resend is missing some detail I included in the original:

> For s390x, I would expect changes similar to c64a10e54441, and
> an eventual similar bug report if this were left unchanged.
> 
> For x86, ppc64 and riscv64, -fpie & -fPIE are identical.

For ppc64 specifically, I audited gcc/config/rs6000/ for references to flag_pic, and the 
only changes between -fpic and -fPIC are for 32-bit ABI_V4.

More generally, our assembly is already set up for -fPIC, which is the pic-ccflag default; 
the difference for -fPIE is only in the selection of external vs local binding.

For aarch64, I've audited uses of the adrp instruction in sysdeps/, and they are what I'd 
expect, e.g.

aarch64/start.S:        adrp    x0, :got:main


and there are no instances of '_GLOBAL_OFFSET_TABLE_', which is indicative of -fpic.

For s390, I've similarly audited uses of the larl instruction:

s390/s390-64/start.S:   larl    %r2,main@GOTENT         # load pointer to main




r~

[RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi,

>> I recall that powerpc wanted to use -mcmodel=large, but it would require to
>> adjust a lot of assembly implementations.  Are we sure this change does not
>> trigger any regression with current glibc code?

I don't see significant differences in terms of codesize on AArch64, so the patch
is fine. As Richard mentioned, it often doesn't make any difference if a function
only uses a single global.

However this exposes a bug: it appears all symbols, including hidden symbols,
are always accessed via the GOT in libc.a even when they don't in libc.so. As a
result PIE code quality is actually worse than PIC... This affects libc_single_threaded
and many uses of internal arrays etc.

So it looks like we should not ignore hidden when building for PIE.

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[H.J. Lu]

On Fri, Aug 26, 2022 at 11:39 AM Wilco Dijkstra via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> Hi,
>
> >> I recall that powerpc wanted to use -mcmodel=large, but it would require to
> >> adjust a lot of assembly implementations.  Are we sure this change does not
> >> trigger any regression with current glibc code?
>
> I don't see significant differences in terms of codesize on AArch64, so the patch
> is fine. As Richard mentioned, it often doesn't make any difference if a function
> only uses a single global.
>
> However this exposes a bug: it appears all symbols, including hidden symbols,
> are always accessed via the GOT in libc.a even when they don't in libc.so. As a
> result PIE code quality is actually worse than PIC... This affects libc_single_threaded

Is this issue processor specific?

> and many uses of internal arrays etc.
>
> So it looks like we should not ignore hidden when building for PIE.



-- 
H.J.

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Richard Henderson]

On 8/26/22 12:02, H.J. Lu wrote:
> On Fri, Aug 26, 2022 at 11:39 AM Wilco Dijkstra via Libc-alpha
>> However this exposes a bug: it appears all symbols, including hidden symbols,
>> are always accessed via the GOT in libc.a even when they don't in libc.so. As a
>> result PIE code quality is actually worse than PIC... This affects libc_single_threaded
> 
> Is this issue processor specific?

I wouldn't think so.


r~

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[H.J. Lu]

On Fri, Aug 26, 2022 at 12:45 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 8/26/22 12:02, H.J. Lu wrote:
> > On Fri, Aug 26, 2022 at 11:39 AM Wilco Dijkstra via Libc-alpha
> >> However this exposes a bug: it appears all symbols, including hidden symbols,
> >> are always accessed via the GOT in libc.a even when they don't in libc.so. As a
> >> result PIE code quality is actually worse than PIC... This affects libc_single_threaded
> >
> > Is this issue processor specific?
>
> I wouldn't think so.
>

This sounds like a compiler bug.

-- 
H.J.

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi,

It's obviously a bug in the way BUILD_PIE_DEFAULT is implemented.
For example one issue is in include/libc-symbols.h:

 #if defined SHARED || defined LIBC_NONSHARED \
  || (BUILD_PIE_DEFAULT && IS_IN (libc))

That should be something like && !IS_IN (nscd) - that builds and removes
a bunch of GOT indirections.

There is a similar issue with special symbol handling, particularly for
__libc_single_threaded.

However this leads to the question is why aren't all symbols marked as hidden
in the static PIE build? Ie. are there any symbols that must be shared with a DSO 
loaded by a dl_open?

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Florian Weimer]

* Wilco Dijkstra via Libc-alpha:

> However this leads to the question is why aren't all symbols marked as
> hidden in the static PIE build? Ie. are there any symbols that must be
> shared with a DSO loaded by a dl_open?

The static main executable does not have a dynamic symbol table, so such
sharing cannot happen (except through function pointers being passed
around, but symbol visibility of course does not matter there).

Thanks,
Florian

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi Florian,

* Wilco Dijkstra via Libc-alpha:
>> However this leads to the question is why aren't all symbols marked as
>> hidden in the static PIE build? Ie. are there any symbols that must be
>> shared with a DSO loaded by a dl_open?
>
> The static main executable does not have a dynamic symbol table, so such
> sharing cannot happen (except through function pointers being passed
> around, but symbol visibility of course does not matter there).

Right, so it should be safe then to mark all symbols as hidden.

We could also teach GCC to never emit GOT indirections with -fPIE -static
(the only reason to use a GOT indirection is to avoid copy relocations in
dynamically linked binaries).

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi Florian,

* Wilco Dijkstra via Libc-alpha:
>> However this leads to the question is why aren't all symbols marked as
>> hidden in the static PIE build? Ie. are there any symbols that must be
>> shared with a DSO loaded by a dl_open?
>
> The static main executable does not have a dynamic symbol table, so such
> sharing cannot happen (except through function pointers being passed
> around, but symbol visibility of course does not matter there).

Right, so it should be safe then to mark all symbols as hidden.

We could also teach GCC to never emit GOT indirections with -fPIE -static
(the only reason to use a GOT indirection is to avoid copy relocations in
dynamically linked binaries).

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[H.J. Lu]

On Mon, Aug 29, 2022 at 8:27 AM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>
> Hi Florian,
>
> * Wilco Dijkstra via Libc-alpha:
> >> However this leads to the question is why aren't all symbols marked as
> >> hidden in the static PIE build? Ie. are there any symbols that must be
> >> shared with a DSO loaded by a dl_open?
> >
> > The static main executable does not have a dynamic symbol table, so such
> > sharing cannot happen (except through function pointers being passed
> > around, but symbol visibility of course does not matter there).
>
> Right, so it should be safe then to mark all symbols as hidden.
>
> We could also teach GCC to never emit GOT indirections with -fPIE -static
> (the only reason to use a GOT indirection is to avoid copy relocations in
> dynamically linked binaries).
>

But -static isn't passed to cc1.


-- 
H.J.

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Fangrui Song]

On 2022-08-29, H.J. Lu via Libc-alpha wrote:
>On Mon, Aug 29, 2022 at 8:27 AM Wilco Dijkstra <Wilco.Dijkstra@arm.com> wrote:
>>
>> Hi Florian,
>>
>> * Wilco Dijkstra via Libc-alpha:
>> >> However this leads to the question is why aren't all symbols marked as
>> >> hidden in the static PIE build? Ie. are there any symbols that must be
>> >> shared with a DSO loaded by a dl_open?
>> >
>> > The static main executable does not have a dynamic symbol table, so such
>> > sharing cannot happen (except through function pointers being passed
>> > around, but symbol visibility of course does not matter there).
>>
>> Right, so it should be safe then to mark all symbols as hidden.
>>
>> We could also teach GCC to never emit GOT indirections with -fPIE -static
>> (the only reason to use a GOT indirection is to avoid copy relocations in
>> dynamically linked binaries).
>>
>
>But -static isn't passed to cc1.

With Clang, -fPIE -fno-direct-access-external-data can be used.
See https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected

GCC's x86 port went with -mdirect-extern-access.

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi,

>>> Right, so it should be safe then to mark all symbols as hidden.
>>>
>>> We could also teach GCC to never emit GOT indirections with -fPIE -static
>>> (the only reason to use a GOT indirection is to avoid copy relocations in
>>> dynamically linked binaries).
>>>
>>
>>But -static isn't passed to cc1.

The driver could just pass it or change -fPIE into a new -fstatic-PIE.

> With Clang, -fPIE -fno-direct-access-external-data can be used.
> See https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
>
> GCC's x86 port went with -mdirect-extern-access.

Well ideally we'd have the same option that works in both compilers - it's likely easy to
support in GCC.

It would be great to also have something like -ffast-PIC that implies -fno-semantic-interposition
since 99% of the time you don't need semantic interposition.

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Fangrui Song]

On 2022-09-05, Wilco Dijkstra wrote:
>Hi,
>
>>>> Right, so it should be safe then to mark all symbols as hidden.
>>>>
>>>> We could also teach GCC to never emit GOT indirections with -fPIE -static
>>>> (the only reason to use a GOT indirection is to avoid copy relocations in
>>>> dynamically linked binaries).
>>>>
>>>
>>>But -static isn't passed to cc1.
>
>The driver could just pass it or change -fPIE into a new -fstatic-PIE.
>
>> With Clang, -fPIE -fno-direct-access-external-data can be used.
>> See https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected
>>
>> GCC's x86 port went with -mdirect-extern-access.
>
>Well ideally we'd have the same option that works in both compilers - it's likely easy to
>support in GCC.
>
>It would be great to also have something like -ffast-PIC that implies -fno-semantic-interposition
>since 99% of the time you don't need semantic interposition.
>
>Cheers,
>Wilco

In Clang, -fpic/-fPIC without  -fno-semantic-interposition still allows
interprocedural optimizations of default visibility external linkage definitions.

It is unfortunate that GCC doesn't allow it and
--enable-default-semantic-interposition is rejected (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937)

So if such an option -ffast-PIC is made, it will only benefit GCC.
I have a long write-up in https://maskray.me/blog/2021-05-09-fno-semantic-interposition#in-action

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi Fangrui,

> In Clang, -fpic/-fPIC without  -fno-semantic-interposition still allows
> interprocedural optimizations of default visibility external linkage definitions.
>
> It is unfortunate that GCC doesn't allow it and
> --enable-default-semantic-interposition is rejected (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937)

I think it now does it with -Ofast, so that's a start at least...

> So if such an option -ffast-PIC is made, it will only benefit GCC.
> I have a long write-up in https://maskray.me/blog/2021-05-09-fno-semantic-interposition#in-action

Well if changing the default of fPIC is unacceptable in GCC, adding a new option
should help LLVM too since it could imply more than -fno-semantic-interposition.
A key question is whether we could require that exported symbols are marked
explicitly so that we only emit GOT and PLT indirections on such exported symbols.

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Florian Weimer]

* Wilco Dijkstra:

> Well if changing the default of fPIC is unacceptable in GCC, adding a
> new option should help LLVM too since it could imply more than
> -fno-semantic-interposition.  A key question is whether we could
> require that exported symbols are marked explicitly so that we only
> emit GOT and PLT indirections on such exported symbols.

-flto with a linker plugin already covers this.  I don't think we need
anything else.  Maybe an LTO mode that only deals with visibility, but
otherwise does little cross-TU optimization?

Thanks,
Florian

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Wilco Dijkstra]

Hi Florian,

>> Well if changing the default of fPIC is unacceptable in GCC, adding a
>> new option should help LLVM too since it could imply more than
>> -fno-semantic-interposition.  A key question is whether we could
>> require that exported symbols are marked explicitly so that we only
>> emit GOT and PLT indirections on such exported symbols.
>
> -flto with a linker plugin already covers this.  I don't think we need
> anything else.  Maybe an LTO mode that only deals with visibility, but
> otherwise does little cross-TU optimization?

Well it would be great if LTO solved this problem, but unfortunately it can't...
LTO tells you which symbols are locally defined, but this doesn't help PIC.
Local definitions with default visibility must still indirect via the GOT/PLT,
similarly interprocedural optimizations are still blocked.

A LTO-lite option that just deals with visibility and global variable optimization
(eg. better use of anchors) and other cheap optimizations seems useful indeed.

Cheers,
Wilco

Re: [RESEND PATCH] Makeconfig: Set pie-ccflag to -fPIE by default

[Fangrui Song]

On 2022-09-20, Wilco Dijkstra wrote:
>Hi Florian,
>
>>> Well if changing the default of fPIC is unacceptable in GCC, adding a
>>> new option should help LLVM too since it could imply more than
>>> -fno-semantic-interposition.  A key question is whether we could
>>> require that exported symbols are marked explicitly so that we only
>>> emit GOT and PLT indirections on such exported symbols.
>>
>> -flto with a linker plugin already covers this.  I don't think we need
>> anything else.  Maybe an LTO mode that only deals with visibility, but
>> otherwise does little cross-TU optimization?

clang -flto={full,thin} respect -fsemantic-interposition.
If -fsemantic-interposition is specified, it disables interprocedural
optimizations (including inlining) for default visibility non-vague external linkage functions.

(
-Ofast unfortunately implies -ffast-math and does crtfastmath.o.
)
Piggybacking -fno-semantic-interposition on -Ofast

>Well it would be great if LTO solved this problem, but unfortunately it can't...
>LTO tells you which symbols are locally defined, but this doesn't help PIC.
>Local definitions with default visibility must still indirect via the GOT/PLT,
>similarly interprocedural optimizations are still blocked.
>
>A LTO-lite option that just deals with visibility and global variable optimization
>(eg. better use of anchors) and other cheap optimizations seems useful indeed.
>
>Cheers,
>Wilco

-fno-semantic-interposition should be sufficient. I think we don't need
another option. We should just the few software to use -fsemantic-interposition so that
the world can be changed.

FreeBSD and quite a few musl-based Linux distributions use Clang as the
system compiler and have verified that -fno-semantic-interposition-like
default (allow interprocedural optimizations for default visibility
non-vague external linkage functions.

I think folks can give
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937 another request.
Even if the upstream doesn't take it, changing a distro's own spec file
to default to -fno-semantic-interposition isn't that difficult.
Many Linux distributions customize their GCC to do some stuff by default
anyway.