From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by sourceware.org (Postfix) with ESMTPS id 4F8A43858404 for ; Thu, 28 Oct 2021 01:06:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4F8A43858404 Received: by mail-pg1-x52d.google.com with SMTP id r2so4681523pgl.10 for ; Wed, 27 Oct 2021 18:06:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=ey3kJ6LQV7qNx+b/K9JtigN+v09oPdd2ZGH5M4kDd/Q=; b=22AGCdncCNW9VaQ0npRygB5mvc0nBx8VloBK2M82yuXSyvNjhf5HQWg1j965lzA4tK YXbJV4ZpQ4nu31QM2uJXlgjihcNKdJXAhVliUFtGEUoPIPIbUe62VMQA1+FPP2EVjSQG Grw/cAd7gfv2xBztep0dIk5qU4xpQ6QgXnVutKAT9TwZGEys5RKRy/+1crBFYnGAwCg2 /iAgUVwm9S89wZnrFXckxJl/q5rbYonH1vrLUA8Ihgteia9lkzVIVGtkMViyB7bdDTKL F9q+vwHos0UW/wPUF3VENXjRnhOM2AWaJQ94X5gQjcv02Y7ESgYmcKV7PzZBrHuyyIlw jmBw== X-Gm-Message-State: AOAM5312lfMy3VoPcotmSaFOwNf8NmFVke5ySfRxXf7unDNaD1pSZznb iQWQ2LrRKFfLUTUL/6DC+zo9NA== X-Google-Smtp-Source: ABdhPJyAsFChGJ8xOj3KrDhllWAEQrWjGJD2nsrMbx4VW8dwQkUsa1JMYbWd+u17a8BqnA7Bnj3UwA== X-Received: by 2002:aa7:93bc:0:b0:47c:1223:3d75 with SMTP id x28-20020aa793bc000000b0047c12233d75mr1271489pff.22.1635383193159; Wed, 27 Oct 2021 18:06:33 -0700 (PDT) Received: from google.com ([2620:15c:2ce:200:5bee:97c3:491b:d4cf]) by smtp.gmail.com with ESMTPSA id m16sm1167543pfk.187.2021.10.27.18.06.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Oct 2021 18:06:32 -0700 (PDT) Date: Wed, 27 Oct 2021 18:06:30 -0700 From: Fangrui Song To: Adhemerval Zanella Cc: libc-alpha@sourceware.org, "H.J. Lu" , Tulio Magno Quites Machado Filho Subject: Re: [PATCH 0/3] Improve lld support and current status Message-ID: <20211028010630.dagff7p6rvygziho@google.com> References: <20211026200346.3371750-1-adhemerval.zanella@linaro.org> <20211026203327.6b2o5k4cmkuzzm6j@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-24.1 required=5.0 tests=BAYES_00, BODY_8BITS, DKIMWL_WL_MED, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH, FSL_HELO_FAKE, GIT_PATCH_0, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLACK, USER_IN_DEF_DKIM_WL, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Oct 2021 01:06:36 -0000 On 2021-10-27, Adhemerval Zanella wrote: > > >On 26/10/2021 17:33, Fangrui Song wrote: >>> The arm, sparcv9, mips, and riscv fail to build due different issues. On >>> arm the loader fails to build: >>> >>>  ld.so fails with >>>  ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol >>>  _dl_argv; recompile with -fPIC >>>  >>> defined in >>>  >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >>>  >>> referenced by rtld.c:164 >>>  >>>               /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) >> >> R_ARM_GOTOFF32 = S + A - GOT_ORG  . >> Such a relocation referencing a preemptible symbol cannot be used. >> >> Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 >> >> _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. > >I think it is something else, the symbols is defined as hidden (output of >preprocessor): > > extern __typeof (_dl_argv) _dl_argv __asm__ ("" "__GI__dl_argv") __attribute__ ((visibility ("hidden"))); > >However, hidden is set only the internal symbol: > > $ arm-glibc-linux-gnueabi-readelf -Ws elf/librtld.os | grep _dl_argv > 1378: 00000000 4 OBJECT GLOBAL DEFAULT 6 _dl_argv > 1470: 00000000 4 OBJECT GLOBAL HIDDEN 6 __GI__dl_argv > >And on rtld.c assembly we have the directive to use the internal definition: > > .set _dl_argv,__GI__dl_argv > >So why lld is not binding the usage to internal to the local hidden alias? I use the Debian package g++-arm-linux-gnueabihf (nowadays glibc build needs a C++ compiler even if I don't build tests...). mkdir -p out/arm; cd out/arm ../../configure --prefix=/tmp/glibc/arm --host=arm-linux-gnueabihf make -j 50 Here is GNU ld produced elf/librtld.os: % readelf -Ws elf/librtld.os | grep _dl_argv 1582: 00000000 4 OBJECT GLOBAL HIDDEN 13 __GI__dl_argv 1603: 00000000 4 OBJECT GLOBAL DEFAULT 13 _dl_argv So LLD's librtld.os matches GNU ld for the two symbols. The .set directive sets value/type but not binding/visibility. Perhaps sysdeps/arm/dl-machine.h should use .word __GI__dl_argv(GOTOFF) instead of .word _dl_argv(GOTOFF) >> >>> On sparcv9, lld handles --relax option different than ld.bfd: it throws >>> an error instead of silent ignoring it: >>> >>>  $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin >>>    -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc >>>  ld.lld: error: unknown argument '-relax' >> >> LLD's sparcv9 port is for retrocomputing fans:) and is far from usable >> (see https://lld.llvm.org/ "production quality"). >> The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). > >Right, but lld is still showing a different ld.bfd semantic that might trigger >other issues where on platforms where neither --relax nor -no-relax it supported >ld.bfd accepts both argument but ignore it [1]. > >At least on sparc, gcc does pass --relax and expects that static linker just >ignore if it is no supported. > >[1] https://sourceware.org/binutils/docs-2.37/ld.html LLD's sparcv9 port is quite immature, so I cannot even advertise it. A large portion should be rewritten but the sparc port is only cared by very few retrocomputing fans now.. >> >>> And even when -mno-relax is explicit add, lld does not support some relocations >>> generated by GCC: >>> >>>  $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args >>>  | head -n2 >>>  00000000000000a0  0000012200000052 R_SPARC_GOTDATA_OP_HIX22 >>>  0000000000000058 _dl_skip_args + 0 >>>  00000000000000a4  0000012200000053 R_SPARC_GOTDATA_OP_LOX10 >>>  0000000000000058 _dl_skip_args + 0 >>> >>> The mips/mipsel also fails to build the loader: >>> >>>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>  '-Wl,-z,notext' to allow text relocations in the output >>>  >>> defined in >>>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) >>> >>>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>  '-Wl,-z,notext' to allow text relocations in the output >>>  >>> defined in >>>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) >> >> lld.llvm.org says "MIPS seems decent too." and I personally know really >> little about MIPS. >> >> However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . >> Relocation processing is rigid and consistent in LLD. >> GNU ld may miss some error checking for some relocation types. >> >>> Similar to mips64/mips64el: >>> >>>  ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>>  _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>>  against non-function symbol . This is invalid and most likely a compiler >>>  bug. >>>  ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>  '-Wl,-z,notext' to allow text relocations in the output >>>  >>> defined in >>>  >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>>  >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>>  >>>               /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >> >> I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >> reporting a genuine issue. > >I give you that loader code is tricky, but it does not really explain why >lld is failing to link the eh_frame. It *might* due some internal assembly >routines, but at least from logs it does not seem so. Hope Joseph can answer the question. If we have easy-to-follow instructions building glibc mips with LLD, I can forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major contributor of LLD's MIPS port)... But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. >> >>> The riscv32/riscv64 fails to the loader, but it is due missing support >>> to relaxation: >>> >>>  ld.lld: error: >>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): >>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>  recompile with -mno-relax >>>  ld.lld: error: >>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): >>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>  recompile with -mno-relax >>>  ld.lld: error: >>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): >>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>  recompile with -mno-rela >> >> LLD does not implement RISC-V linker relaxation. >> R_RISCV_ALIGN has weird linker unfriendly semantics: >> simply ignoring the relocation type can break semantics >> (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). >> >>> Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. >> >> So sysdep-CFLAGS is not a catch-all option affecting all C compiles... > >In fact it is, the problem was the autogenerated syscalls built from >the assembly. I could avoid the R_RISCV_ALIGN with: > >diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile >index 20a9968106..28dbd89cb0 100644 >--- a/sysdeps/riscv/Makefile >+++ b/sysdeps/riscv/Makefile >@@ -5,3 +5,9 @@ endif > # RISC-V's assembler also needs to know about PIC as it changes the definition > # of some assembler macros. > ASFLAGS-.os += $(pic-ccflag) >+ >+# lld does not implement R_RISCV_ALIGN relaxation optimization. >+ifeq (yes,$(with-lld)) >+ASFLAGS-.os += -Wa,-mno-relax >+sysdep-CFLAGS += -mno-relax >+endif > > >However it does fail later for the libc.so with: > >ld.lld: warning: attempt to reassign symbol 'fcntl' of version 'GLIBC_2.27' to version 'GLIBC_2.28' >ld.lld: warning: attempt to reassign symbol '__sched_get_priority_min' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >ld.lld: warning: attempt to reassign symbol '__sched_get_priority_max' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >ld.lld: error: relocation R_RISCV_RVC_JUMP cannot be used against symbol __sigsetjmp; recompile with -fPIC >>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>> referenced by iconv.c:62 (/home/azanella/Projects/glibc/glibc-git/iconv/iconv.c:62) >>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__GI__setjmp) > >ld.lld: error: relocation R_RISCV_JAL cannot be used against symbol exit; recompile with -fPIC >>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>> referenced by gconv_open.c:89 (/home/azanella/Projects/glibc/glibc-git/iconv/gconv_open.c:89) >>>> /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__start_context) There is a GNU ld issue: https://sourceware.org/bugzilla/show_bug.cgi?id=28509 It incorrectly allows non-PLT non-GOT relocation referencing a preemptible symbol. LLD uses consistent categories (e.g. R_PC / R_PLT_PC) to describe relocation types. It has consistent diagnostic across all ports it supports. GNU ld uses dispatches for each relocation type and may miss some diagnostics. >collect2: error: ld returned 1 exit status