[Bug driver/100937] New: configure: Add --enable-default-semantic-interposition

[Bug driver/100937] New: configure: Add --enable-default-semantic-interposition

[i at maskray dot me]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

            Bug ID: 100937
           Summary: configure: Add --enable-default-semantic-interposition
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: driver
          Assignee: unassigned at gcc dot gnu.org
          Reporter: i at maskray dot me
  Target Milestone: ---

Add a configure option --enable-default-semantic-interposition to customize
-f(no-)semantic-interposition default.

The suppression of interprocedural optimizations and inlining for such
default visibility non-vague-linkage function definitions is the biggest
difference between -fPIE/-fPIC.

Distributions may want to enable default -fno-semantic-interposition to
reclaim the lost performance from -fPIC (e.g. CPython is said to be 27% faster;
Clang is 3% faster).

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WONTFIX
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #1 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
NO. This is wrong for many reasons. First it makes portability a pain.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[i at maskray dot me]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

Fangrui Song <i at maskray dot me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|WONTFIX                     |---
             Status|RESOLVED                    |UNCONFIRMED

--- Comment #2 from Fangrui Song <i at maskray dot me> ---
How is it a portability problem?

clang -fpic has always been allowing interprocedural optimizations for
non-vague-linkage function definitions. FreeBSD uses clang and software works
with no problem.




For a vague-linkage function definition, a call site in the same
translation unit may inline the callee. Whether
-fno-semantic-interposition is enabled/disabled has no effect.

For a non-vague-linkage function definition, by default
(-fsemantic-interposition) the -fpic mode does not allow a call site
in the same translation unit to inline the callee or perform other
interprocedural optimizations.
-fno-semantic-interposition re-enables interprocedural optimizations.

If a caller inlines a callee, using LD_PRELOAD to interpose the callee
will not affect the caller. But many other LD_PRELOAD usage still
work.
We consider the small LD_PRELOAD limitation a good trade off for the speedup.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |WONTFIX
             Status|UNCONFIRMED                 |RESOLVED

--- Comment #3 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
>clang -fpic has always been allowing interprocedural optimizations for non-vague-linkage function definitions. FreeBSD uses clang and software works with no problem.

That does not mean clang is correct here.
clang breaks ELF assumptions and that is all I am going to say.  If you want to
break ELF fine, FreeBSD can break those.  But there is still a portability
issue between distros using different options like this.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #4 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
Also your patch did not change the documentation of the option.
Plus the documentation is clear that changing the default is most likely not
wanted at all:
https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Optimize-Options.html#index-fsemantic-interposition

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

Jakub Jelinek <jakub at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #5 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
Agreed this is a very bad idea.  Users can always pick options they want
explicitly, but changing such options' defaults will mean nothing will be able
to rely on the default and so everything that will need semantic interposition
will all of sudden need to start using -fsemantic-interposition.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[i at maskray dot me]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #6 from Fangrui Song <i at maskray dot me> ---
Then can you add a -fvisibility=protected variant which only applies to
non-weak defined functions?

Two issues need to be fixed:

(1): https://sourceware.org/bugzilla/show_bug.cgi?id=27973

__attribute__((visibility("protected"))) void *foo () {
  return (void *)foo;
}
% gcc -fpic -shared -fuse-ld=bfd a.s
/usr/bin/ld.bfd: /tmp/ccWPJCLw.o: relocation R_X86_64_PC32 against protected
symbol `foo' can not be used when making a shared object
/usr/bin/ld.bfd: final link failed: bad value
collect2: error: ld returned 1 exit status

(2): https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100593
[ELF] -fno-pic: Use GOT to take address of an external default visibility
function 


Distributions want fast C++ non-vague-linkage functions can enable this option.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[frankhb1989 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

frankhb1989 at gmail dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |frankhb1989 at gmail dot com

--- Comment #7 from frankhb1989 at gmail dot com ---
While I feel it is fair to keep the status quo, the decision here deserves some
additional comments.

It is true that potability is generally important, but the current way here is
actually not friendly to potability.

GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to the
authority of the specification. The "ELF assumptions" have no natural position
to be the default at the very first glance from the users' view.

So the "portability" certainly include the ease of porting the programs to
different targets with different image formats back and forth. Sticking on the
ELF-centric defaults already fails (by reducing the platform-specific
differences) if users want to really gain more portability, at least for
PE/COFF targets (which do not support such interposition at all).

Although it is somewhat reasonable to distinguish platforms supporting symbol
interposition as first-class ones (same to the current "primary platforms"),
but this still seems technically weak.

Perhaps a more appropriate phrase for the reason is "backward compatibility".

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

Eric Gallager <egallager at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=102665
                 CC|                            |egallager at gcc dot gnu.org

--- Comment #8 from Eric Gallager <egallager at gcc dot gnu.org> ---
(In reply to frankhb1989 from comment #7) 
> GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to
> the authority of the specification. The "ELF assumptions" have no natural
> position to be the default at the very first glance from the users' view.
> 
> So the "portability" certainly include the ease of porting the programs to
> different targets with different image formats back and forth. Sticking on
> the ELF-centric defaults already fails (by reducing the platform-specific
> differences) if users want to really gain more portability, at least for
> PE/COFF targets (which do not support such interposition at all).

This is related to bug 102665 btw

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[redi at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #9 from Jonathan Wakely <redi at gcc dot gnu.org> ---
(In reply to frankhb1989 from comment #7)
> GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to
> the authority of the specification.

That seems like nonsense to me. The toolchain might not be ELF-specific, but on
targets that *do* use ELF, of course the ELF specification is relevant, and
deviating from it should not be done lightly.

>  The "ELF assumptions" have no natural
> position to be the default at the very first glance from the users' view.

More nonsense.

Just because portability between ELF and PE/COFF is difficult and requires
care, doesn't mean we should also make it difficult to guarantee portability
across different linux distros that are all using ELF. That's just silly. You
don't make anything better by making everything equally bad.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[frankhb1989 at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #10 from frankhb1989 at gmail dot com ---
(In reply to Jonathan Wakely from comment #9)
> (In reply to frankhb1989 from comment #7)
> The toolchain might not be ELF-specific, but
> on targets that *do* use ELF, of course the ELF specification is relevant,

True.

> and deviating from it should not be done lightly.

Deviating... well, are there any documented policies to clarify how much room
to the non-conformance here?

> Just because portability between ELF and PE/COFF is difficult and requires
> care, doesn't mean we should also make it difficult to guarantee portability
> across different linux distros that are all using ELF. That's just silly.
> You don't make anything better by making everything equally bad.

I don't mean that one portability is just unconditionally more important than
another. But there are certainly things beyond portability if you are really
caring "everything".

Keeping the "portability", if any across ELF, makes sense to maintainers of the
distros.

But it is not obvious to most users, which are not specialists of ELF. For
those not relying on the sneaky feature, they silently pay the unneeded cost
*by default*. This is probably against to the intuition to most C and C++
users.

Moreover, this also affects almost every *end users*, which should not care
about such features at all.

So here is the "bad": the default configuration makes things unexpected (and a
laborious way to prevent it) for most users, albeit only in QoI sense. Such
default may even have effects on the adoption of free software.

Changing the default may likely reduce such bad things, at the cost of the
compatibility. I am not arguing the change is absolutely necessary, but it is
certainly NOT "making everything equally bad". (This would not remove
interposition for the minority who need them indeed.)

BTW, I'm not aware of such a cross-distro guarantee has been widely adopted and
relied on. It is well-known the Linux kernel has the policy to maintain the
stability of syscalls, but in the userland... is it already distro-specific?
Correct me if I am wrong here. (Any source?)

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[i at maskray dot me]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #11 from Fangrui Song <i at maskray dot me> ---
To enable interposition on Mach-O, one needs a non-default configuration like:
ld -interposable, DYLD_FORCE_FLAT_NAMESPACE or
__attribute__((section("__DATA,__interpose"))).
On PE/COFF, such interposition just doesn't exist.

Having an option for -fno-semantic-interposition will actually improve
portability.

(The -fno-semantic-interposition thing is probably the biggest performance gap
between gcc -fpic and clang -fpic.)

As I said previously, -fvisibility=protected cannot be used because protected
visibility is very broken in the GCC/GNU ld system and there is no signal it
will be fixed anytime soon:
https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#summary

Re: [Bug driver/100937] configure: Add --enable-default-semantic-interposition

[Jan Hubicka]

> (The -fno-semantic-interposition thing is probably the biggest performance gap
> between gcc -fpic and clang -fpic.)
Yep, it is often confusing to users (who do not understand what ELF
interposition is) that clang and gcc disagree on default flags here.
Recently -Ofast was extended to imply -fno-semantic-interposition that
will hopefully make more people notice this.

While doing that I have added per-symbol flag about interposition to the
symbol table, so we can also support 

__atttribute__ ((semantic_interposition))

and

__attribute__((no_semantic_interpoition))

if that would be useful for something.

[Bug driver/100937] configure: Add --enable-default-semantic-interposition

[hubicka at kam dot mff.cuni.cz]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100937

--- Comment #12 from hubicka at kam dot mff.cuni.cz ---
> (The -fno-semantic-interposition thing is probably the biggest performance gap
> between gcc -fpic and clang -fpic.)
Yep, it is often confusing to users (who do not understand what ELF
interposition is) that clang and gcc disagree on default flags here.
Recently -Ofast was extended to imply -fno-semantic-interposition that
will hopefully make more people notice this.

While doing that I have added per-symbol flag about interposition to the
symbol table, so we can also support 

__atttribute__ ((semantic_interposition))

and

__attribute__((no_semantic_interpoition))

if that would be useful for something.