Re: [PATCH 0/3] Improve lld support and current status

[Fangrui Song <maskray@google.com>]

On 2021-10-26, Adhemerval Zanella wrote:
>Ths patch allows x86_64, i686, aarch64, powerpc64le, powerpc64, and
>powerpc to build, including the tests, with LLD packed in LLVM 13 [1].

Thanks for checking!

>For x86_64 and aarch64 there is no regression in testssuite.  On
>i686 there is only one:
>
>  FAIL: elf/ifuncmain6pie
>
>Which segfaults calling the 'foo' functions:
>
>  Program received signal SIGSEGV, Segmentation fault.
>  0xfb010101 in ?? ()
>  (gdb) bt
>  #0  0xfb010101 in ?? ()
>  #1  0xf7fba620 in call_foo () at ifuncmod6.c:18
>  #2  0xf7fc0857 in ?? ()
>  #3  0xf7e1b195 in __libc_start_call_main (main=main@entry=0xf7fc0830,
>  argc=argc@entry=2, argv=argv@entry=0xffffc610) at
>  ../sysdeps/nptl/libc_start_call_main.h:58
>  #4  0xf7e1b286 in __libc_start_main_impl (main=0xf7fc0830, argc=2,
>  argv=0xffffc610, init=0x0, fini=0x0, rtld_fini=0xf7fe79b0 <_dl_fini>,
>  stack_end=0xffffc60c) at ../csu/libc-start.c:409
>  #5  0xf7fc06d7 in ?? ()
>
>The powerpc64le fails at libc.so start:
>
>  (gdb) bt
>  #0  0x00007ffff7f2a980 in __gep_setup___vmx__sigjmp_save ()
>    from
>  /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6
>  #1  0x00007ffff7d7ac5c in __libc_start_call_main ()
>     from
>  /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6
>  #2  0x00007ffff7d7ae90 in __libc_start_main_impl ()
>     from
>  /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6
>  #3  0x0000000000000000 in ?? ()
>  (gdb) disas
>  Dump of assembler code for function __gep_setup___vmx__sigjmp_save:
>  => 0x00007ffff7f2a980 <+0>:     .long 0x613ffe6
>     0x00007ffff7f2a984 <+4>:     li      r12,-1280
>     0x00007ffff7f2a988 <+8>:     mtctr   r12
>     0x00007ffff7f2a98c <+12>:    bctr
>
>And powerpc64-linux-gnu fails on dynamic loaders start:
>
>  (gdb) bt
>  #0  0x00007ffff7ffa9a8 in _start ()
>  (gdb) disas
>  Dump of assembler code for function _start:
>  => 0x00007ffff7ffa9a8 <+0>:     .long 0x0
>     0x00007ffff7ffa9ac <+4>:     .long 0x0
>     0x00007ffff7ffa9b0 <+8>:     .long 0x0
>     0x00007ffff7ffa9b4 <+12>:    .long 0x0

CCed the powerpc maintainer.

I am not familiar with debugging rtld for powerpc64...

>
>The arm, sparcv9, mips, and riscv fail to build due different issues. On
>arm the loader fails to build:
>
>  ld.so fails with
>  ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol
>  _dl_argv; recompile with -fPIC
>  >>> defined in
>  >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os
>  >>> referenced by rtld.c:164
>  >>>               /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8)

R_ARM_GOTOFF32 = S + A - GOT_ORG  .
Such a relocation referencing a preemptible symbol cannot be used.

Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501

_dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility.

>On sparcv9, lld handles --relax option different than ld.bfd: it throws
>an error instead of silent ignoring it:
>
>  $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin
>    -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc
>  ld.lld: error: unknown argument '-relax'

LLD's sparcv9 port is for retrocomputing fans:) and is far from usable
(see https://lld.llvm.org/ "production quality").
The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985).

>And even when -mno-relax is explicit add, lld does not support some relocations
>generated by GCC:
>
>  $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args
>  | head -n2
>  00000000000000a0  0000012200000052 R_SPARC_GOTDATA_OP_HIX22
>  0000000000000058 _dl_skip_args + 0
>  00000000000000a4  0000012200000053 R_SPARC_GOTDATA_OP_LOX10
>  0000000000000058 _dl_skip_args + 0
>
>The mips/mipsel also fails to build the loader:
>
>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local
>  symbol in readonly segment; recompile object files with -fPIC or pass
>  '-Wl,-z,notext' to allow text relocations in the output
>  >>> defined in
>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os
>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c
>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C)
>
>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local
>  symbol in readonly segment; recompile object files with -fPIC or pass
>  '-Wl,-z,notext' to allow text relocations in the output
>  >>> defined in
>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os
>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c
>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54)

lld.llvm.org says "MIPS seems decent too." and I personally know really
little about MIPS.

However, I think this is likely a genuine issue somewhere in sysdeps/mips/ .
Relocation processing is rigid and consistent in LLD.
GNU ld may miss some error checking for some relocation types.

>Similar to mips64/mips64el:
>
>  ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function
>  _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation
>  against non-function symbol . This is invalid and most likely a compiler
>  bug.
>  ld.lld: error: can't create dynamic relocation R_MIPS_64 against local
>  symbol in readonly segment; recompile object files with -fPIC or pass
>  '-Wl,-z,notext' to allow text relocations in the output
>  >>> defined in
>  >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os
>  >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c
>  >>>               /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20)

I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it
reporting a genuine issue.

>The riscv32/riscv64 fails to the loader, but it is due missing support
>to relaxation:
>
>  ld.lld: error:
>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84):
>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
>  recompile with -mno-relax
>  ld.lld: error:
>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC):
>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
>  recompile with -mno-relax
>  ld.lld: error:
>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE):
>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation;
>  recompile with -mno-rela

LLD does not implement RISC-V linker relaxation.
R_RISCV_ALIGN has weird linker unfriendly semantics:
simply ignoring the relocation type can break semantics
(https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183).

>Adding -mno-relax to sysdep-CFLAGS seems to be ineffective.

So sysdep-CFLAGS is not a catch-all option affecting all C compiles...

>
>[1] https://releases.llvm.org/download.html#13.0.0
>
>Adhemerval Zanella (3):
>  elf: Disable ifuncmain{1,5,5pic,5pie} when using LLD
>  Fix LIBC_PROG_BINUTILS for -fuse-ld=lld
>  Check if linker also support -mtls-dialect=gnu2
>
> aclocal.m4   |  6 +++++-
> configure    | 10 +++++++---
> configure.ac |  4 ++--
> elf/Makefile | 17 +++++++++++++----
> 4 files changed, 27 insertions(+), 10 deletions(-)
>
>-- 
>2.32.0
>