Re: lld status with powerpc64

[Adhemerval Zanella <adhemerval.zanella@linaro.org>]

On 08/11/2021 19:38, Fangrui Song wrote:
> I agree with Adhemerval's point that "The main problem is this imposes
> an extra burden for the linker" is unfortunate. (Magic behavior of
> --power10-stubs=auto).

That the main problem in fact, where ld.bfd and lld have different semantics
for --power10-stubs=auto and it is the default option. 

> 
> On 2021-11-08, Nemanja Ivanovic wrote:
>> Just to chime in here...
>>
>> Adhemerval,
>> just a couple of notes about what you are referring to as an "optimization".
>> The reason that LLD emits P10 PC-Relative prefixed instructions in stubs is
>> because of an implicit assumption that the *_NOTOC relocation is restricted for
>> P10 PC-Relative builds as it was never emitted by any compiler prior. However,
>> I don't think it is accurate to refer to no P10 stubs with *_NOTOC as an
>> "optimization". The two linkers have slightly different logic for deciding when
>> it is OK to use P10 stubs. GLIBC is developed using the GNU toolchain so
>> naturally relies on behaviour of GNU tools. In this case, the GNU linkers do
>> not emit P10 stubs so GLIBC decided to use the @notoc. However, in a slightly
>> different situation, the GNU linkers would generate P10 stubs as well. I
>> imagine if GLIBC development added code that causes that to happen, that code
>> would never be upstreamed as it causes broken builds.

I agree that glibc is tied to GNU toolchain (we aim to use 'GNU C' along with
all its extension as the base language) and I also agree that such optimization
might be costly for lld.

But from an user perspective I would expect that if ldd is trying to mimic 
ld.bfd behavior regarding power10 stub creation, the semantic should be at least 
the same or error if something is not supported. The issue is exactly that subtle
issues that arise when something is either underspecified or not properly documented.

>>
>> Ultimately, what is needed is a clear indication in each relocation that using
>> P10 stubs is OK for that call. Relying on presence of P10 relocations elsewhere
>> in the compilation unit is problematic for at least two reasons (less efficient
>> for the linker to look over all relocations twice, can lead to incorrect
>> choices). This is why the proposal is to have different relocations for P10 and
>> pre-P10 code. My preference would be to make this clear in the textual
>> relocations as well (rather than the assembler emitting two different
>> relocations for @notoc depending on -mpower10 vs -m<something_older>).
> 
> Thanks for investigating the long-term solution.
> 
> Nowadays among popular architectures, all of AArch64/RISC-V/x86 have
> open places discussing ABI and toolchain implementations. It'd be nice
> if ppc64 had a similar place so that we could detect problems earlier.
> If more eyes had stared at the @notoc usage, we probably could have
> avoided some friction in glibc and ld.lld...
> 
> ---
> 
>> Current course and speed is that @notoc will imply pcrel stubs on P10
>> and later, but will not do so on P9 and earlier.  The relocation
>> currently associated with @notoc actually came with ELFv2 on P8 and,
>> although no compilers ever generated @notoc, assembly routines using it
>> prior to P10 are a valid case and should not be punished.  This can be
>> handled by generating a different reloc for @notoc in the two cases.
> 
> In the context of glibc's @notoc assembly usage, I don't see why
> suppresing @notoc for pre-POWER10 builds  would be a punishment:)
> 
> The branch targets are non-preemptible symbols.
> In the absence of a long branch, @notoc to toc code needs a thunk (in
> both GNU ld and ld.lld) while toc to toc doesn't.
> 
> In the case of a long branch, @notoc with PC-relative instructions is
> slightly more efficient because it uses PC-relative instead of TOC's loading an address from .branch_lt
> but the very minor improvement doesn't seem to justify the complexity :)
> 
> I say "minor" because I have checked the glibc build on a POWER9 machine
> (GCC 8.3.0, binutils 2.31.1) and an x86-64 using cross compilatioin (GCC
> 10.2.1, binutils 2.37).
> On the POWER9 machine, I don't find any thunk for __syscall_error and a
> few other symbols using the glibc NOTOC macro.
> On x86-64, I saw a long branch call site but I am not sure it can ever
> be hot :)

I won't indeed, since @notoc only is helpful when power8 objects are being
created.

> 
> For
> https://sourceware.org/pipermail/libc-alpha/2021-November/132767.html ,
> I'd like to support it. It will address the immediate problem linking
> glibc with LLD 13.0.0 since glibc 2.35 is not too faraway (scheduled on
> 2022-02-01).
> 
> But the patch seems to complicate things.
> I think checking whether with the supplied CFLAGS, CC generates @notoc
> is better than checking assembler support.

Well, that is exactly of the patch does:

  libc_cv_ppc64_notoc=no
  if AC_TRY_COMMAND([${CC-cc} $CFLAGS $CPPFLAGS -S -o conftest.s conftest.c]) \
     && AC_TRY_COMMAND([grep -q -E 'bar@notoc' conftest.s])
  then
    libc_cv_ppc64_notoc=yes
  fi

I think the title is misleading since it tests only compiler, not linker.