From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk1-x72d.google.com (mail-qk1-x72d.google.com [IPv6:2607:f8b0:4864:20::72d]) by sourceware.org (Postfix) with ESMTPS id B5D133858410 for ; Thu, 28 Oct 2021 11:48:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org B5D133858410 Received: by mail-qk1-x72d.google.com with SMTP id bm16so5396118qkb.11 for ; Thu, 28 Oct 2021 04:48:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=w7F4Iz8xmAin3ekgHWTAfFpg4bmYFsIAI8sit9pUbwM=; b=vjv3sAbIpXJ0fNXEHKIidnnGHk4nX/Hw6gd20pPaKy0rrIHlq0BuwJPfbEeKs82iFW wA3vrIf8L6SmXBzpc5AtBorFriQIRnJ9QFbWpgYudkojqg48Fwl4kBDNtCT2BuOvX+5X 4gtMrQcBLjWayl6l1nUmZ47d5JJulu4jLBVfeq1uD3Z0rrou7J7Etaow6vt1zebKxdIQ Cr9p1DPfbYivVgKlJysEcS95VFtxy9h2gEHchj1QvkUkV/0x3Q6fk30imMguNSHRwDWK 3OQYz1DQ4Iq5tghf7IqNaDLCAhZGYzzV9biD9NfEthZ8jbNbYLu1cOoBHbbIZsmMsP4o skeQ== X-Gm-Message-State: AOAM532eao22FAkCJXDNmJJlW18zJC6OolzIOCPYBP9H4p2/0KyCNh4v HOXEYiORqllX9cJFxQrzlEq2Mw== X-Google-Smtp-Source: ABdhPJw0F4uhuuIww22fLZoYrxwYcGzoTavVbzV8li23y10b2K/VQ74dZlFOtfWpmZd1ZrcRYWxjlg== X-Received: by 2002:a37:668b:: with SMTP id a133mr3055461qkc.91.1635421716146; Thu, 28 Oct 2021 04:48:36 -0700 (PDT) Received: from ?IPV6:2804:431:c7cb:b64f:597e:dd64:5db8:5deb? ([2804:431:c7cb:b64f:597e:dd64:5db8:5deb]) by smtp.gmail.com with ESMTPSA id x7sm1777608qko.109.2021.10.28.04.48.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 28 Oct 2021 04:48:35 -0700 (PDT) Message-ID: Date: Thu, 28 Oct 2021 08:48:33 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.2 Subject: Re: [PATCH 0/3] Improve lld support and current status Content-Language: en-US To: Fangrui Song Cc: libc-alpha@sourceware.org, "H.J. Lu" , Tulio Magno Quites Machado Filho References: <20211026200346.3371750-1-adhemerval.zanella@linaro.org> <20211026203327.6b2o5k4cmkuzzm6j@google.com> <20211028010630.dagff7p6rvygziho@google.com> From: Adhemerval Zanella In-Reply-To: <20211028010630.dagff7p6rvygziho@google.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.3 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP, URIBL_BLACK autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Oct 2021 11:48:40 -0000 On 27/10/2021 22:06, Fangrui Song wrote: > > On 2021-10-27, Adhemerval Zanella wrote: >> >> >> On 26/10/2021 17:33, Fangrui Song wrote: >>>> The arm, sparcv9, mips, and riscv fail to build due different issues. On >>>> arm the loader fails to build: >>>> >>>>  ld.so fails with >>>>  ld.lld: error: relocation R_ARM_GOTOFF32 cannot be used against symbol >>>>  _dl_argv; recompile with -fPIC >>>>  >>> defined in >>>>  >>> /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os >>>>  >>> referenced by rtld.c:164 >>>>  >>>               /home/azanella/Projects/glibc/build/arm-linux-gnueabihf-lld/elf/librtld.os:(.text+0xA8) >>> >>> R_ARM_GOTOFF32 = S + A - GOT_ORG  . >>> Such a relocation referencing a preemptible symbol cannot be used. >>> >>> Filed GNU ld bug: https://sourceware.org/bugzilla/show_bug.cgi?id=28501 >>> >>> _dl_argv needs to be made non-preemptible. There are many ways: --dynamic-list, -Bsymbolic, hidden visibility. >> >> I think it is something else, the symbols is defined as hidden (output of >> preprocessor): >> >>  extern __typeof (_dl_argv) _dl_argv __asm__ ("" "__GI__dl_argv") __attribute__ ((visibility ("hidden"))); >> >> However, hidden is set only the internal symbol: >> >>  $ arm-glibc-linux-gnueabi-readelf -Ws elf/librtld.os | grep _dl_argv >>    1378: 00000000     4 OBJECT  GLOBAL DEFAULT    6 _dl_argv >>    1470: 00000000     4 OBJECT  GLOBAL HIDDEN     6 __GI__dl_argv >> >> And on rtld.c assembly we have the directive to use the internal definition: >> >>    .set    _dl_argv,__GI__dl_argv >> >> So why lld is not binding the usage to internal to the local hidden alias? > > I use the Debian package g++-arm-linux-gnueabihf (nowadays glibc build needs a C++ compiler even if I don't build tests...). You can set an empty CXX to avoid it ("CXX=") at configure time. > > mkdir -p out/arm; cd out/arm > ../../configure --prefix=/tmp/glibc/arm --host=arm-linux-gnueabihf > make -j 50 > > Here is GNU ld produced elf/librtld.os: > > % readelf -Ws elf/librtld.os | grep _dl_argv >   1582: 00000000     4 OBJECT  GLOBAL HIDDEN    13 __GI__dl_argv >   1603: 00000000     4 OBJECT  GLOBAL DEFAULT   13 _dl_argv > > So LLD's librtld.os matches GNU ld for the two symbols. > > > > The .set directive sets value/type but not binding/visibility. > Perhaps sysdeps/arm/dl-machine.h should use > > .word   __GI__dl_argv(GOTOFF) > > instead of > .word   _dl_argv(GOTOFF) > It think it should be ok. So it seems that ld.bfd is being forgiving here and binding to the internal alias. >>> >>>> On sparcv9, lld handles --relax option different than ld.bfd: it throws >>>> an error instead of silent ignoring it: >>>> >>>>  $ sparc64-glibc-linux-gnu-gcc -mcpu=niagara -fuse-ld=lld -Bclang+llvm-13.0.0-x86_64-linux-gnu-ubuntu-20.04/bin >>>>    -g -O2 -fPIC -shared -fno-stack-protector -o conftest.so conftest.c -nostdlib -nostartfiles -Wl,-z,combreloc >>>>  ld.lld: error: unknown argument '-relax' >>> >>> LLD's sparcv9 port is for retrocomputing fans:) and is far from usable >>> (see https://lld.llvm.org/ "production quality"). >>> The GOT/PLT support has quite a few issues (https://reviews.llvm.org/D102985). >> >> Right, but lld is still showing a different ld.bfd semantic that might trigger >> other issues where on platforms where neither --relax nor -no-relax it supported >> ld.bfd accepts both argument but ignore it [1]. >> >> At least on sparc, gcc does pass --relax and expects that static linker just >> ignore if it is no supported. >> >> [1] https://sourceware.org/binutils/docs-2.37/ld.html > > LLD's sparcv9 port is quite immature, so I cannot even advertise it. > A large portion should be rewritten but the sparc port is only cared by very few > retrocomputing fans now.. It would be good if we could detect it at configure time and just warn that linker is not sufficient to build glibc. > >>> >>>> And even when -mno-relax is explicit add, lld does not support some relocations >>>> generated by GCC: >>>> >>>>  $ sparc64-glibc-linux-gnu-readelf -Wr elf/librtld.os | grep _dl_skip_args >>>>  | head -n2 >>>>  00000000000000a0  0000012200000052 R_SPARC_GOTDATA_OP_HIX22 >>>>  0000000000000058 _dl_skip_args + 0 >>>>  00000000000000a4  0000012200000053 R_SPARC_GOTDATA_OP_LOX10 >>>>  0000000000000058 _dl_skip_args + 0 >>>> >>>> The mips/mipsel also fails to build the loader: >>>> >>>>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>>  '-Wl,-z,notext' to allow text relocations in the output >>>>  >>> defined in >>>>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x1C) >>>> >>>>  ld.lld: error: can't create dynamic relocation R_MIPS_32 against local >>>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>>  '-Wl,-z,notext' to allow text relocations in the output >>>>  >>> defined in >>>>  >>> /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os >>>>  >>> referenced by /home/azanella/toolchain/src/gcc/libgcc/libgcc2.c >>>>  >>>               /home/azanella/Projects/glibc/build/mips-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x54) >>> >>> lld.llvm.org says "MIPS seems decent too." and I personally know really >>> little about MIPS. >>> >>> However, I think this is likely a genuine issue somewhere in sysdeps/mips/ . >>> Relocation processing is rigid and consistent in LLD. >>> GNU ld may miss some error checking for some relocation types. >>> >>>> Similar to mips64/mips64el: >>>> >>>>  ld.lld: warning: ../sysdeps/unix/sysv/linux/setitimer.c:(function >>>>  _dl_runtime_resolve: .text+0x18954): found R_MIPS_JALR relocation >>>>  against non-function symbol . This is invalid and most likely a compiler >>>>  bug. >>>>  ld.lld: error: can't create dynamic relocation R_MIPS_64 against local >>>>  symbol in readonly segment; recompile object files with -fPIC or pass >>>>  '-Wl,-z,notext' to allow text relocations in the output >>>>  >>> defined in >>>>  >>> /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os >>>>  >>> referenced by ../sysdeps/unix/sysv/linux/setitimer.c >>>>  >>>               /home/azanella/Projects/glibc/build/mips64-linux-gnu-lld/elf/librtld.os:(.eh_frame+0x20) >>> >>> I took a look at LLD's R_MIPS_JALR code and I am inclined to trust it >>> reporting a genuine issue. >> >> I give you that loader code is tricky, but it does not really explain why >> lld is failing to link the eh_frame.  It *might* due some internal assembly >> routines, but at least from logs it does not seem so. > > Hope Joseph can answer the question. > > If we have easy-to-follow instructions building glibc mips with LLD, I can > forward it to Simon Atanasyan (MIPS code owner of llvm-project and the major > contributor of LLD's MIPS port)... > > But see below (for riscv/), I suspect this is a glibc sysdeps/ problem. > >>> >>>> The riscv32/riscv64 fails to the loader, but it is due missing support >>>> to relaxation: >>>> >>>>  ld.lld: error: >>>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2584:(.text+0xEC84): >>>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>>  recompile with -mno-relax >>>>  ld.lld: error: >>>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEC): >>>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>>  recompile with -mno-relax >>>>  ld.lld: error: >>>>  /home/azanella/Projects/glibc/glibc-git/elf/rtld.c:2589:(.text+0x13DEE): >>>>  relocation R_RISCV_ALIGN requires unimplemented linker relaxation; >>>>  recompile with -mno-rela >>> >>> LLD does not implement RISC-V linker relaxation. >>> R_RISCV_ALIGN has weird linker unfriendly semantics: >>> simply ignoring the relocation type can break semantics >>> (https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/183). >>> >>>> Adding -mno-relax to sysdep-CFLAGS seems to be ineffective. >>> >>> So sysdep-CFLAGS is not a catch-all option affecting all C compiles... >> >> In fact it is, the problem was the autogenerated syscalls built from >> the assembly.  I could avoid the R_RISCV_ALIGN with: >> >> diff --git a/sysdeps/riscv/Makefile b/sysdeps/riscv/Makefile >> index 20a9968106..28dbd89cb0 100644 >> --- a/sysdeps/riscv/Makefile >> +++ b/sysdeps/riscv/Makefile >> @@ -5,3 +5,9 @@ endif >> # RISC-V's assembler also needs to know about PIC as it changes the definition >> # of some assembler macros. >> ASFLAGS-.os += $(pic-ccflag) >> + >> +# lld does not implement R_RISCV_ALIGN relaxation optimization. >> +ifeq (yes,$(with-lld)) >> +ASFLAGS-.os += -Wa,-mno-relax >> +sysdep-CFLAGS += -mno-relax >> +endif >> >> >> However it does fail later for the libc.so with: >> >> ld.lld: warning: attempt to reassign symbol 'fcntl' of version 'GLIBC_2.27' to version 'GLIBC_2.28' >> ld.lld: warning: attempt to reassign symbol '__sched_get_priority_min' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >> ld.lld: warning: attempt to reassign symbol '__sched_get_priority_max' of version 'GLIBC_2.27' to version 'GLIBC_PRIVATE' >> ld.lld: error: relocation R_RISCV_RVC_JUMP cannot be used against symbol __sigsetjmp; recompile with -fPIC >>>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>>> referenced by iconv.c:62 (/home/azanella/Projects/glibc/glibc-git/iconv/iconv.c:62) >>>>>               /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__GI__setjmp) >> >> ld.lld: error: relocation R_RISCV_JAL cannot be used against symbol exit; recompile with -fPIC >>>>> defined in /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean >>>>> referenced by gconv_open.c:89 (/home/azanella/Projects/glibc/glibc-git/iconv/gconv_open.c:89) >>>>>               /home/azanella/Projects/glibc/build/riscv64-linux-gnu-rv64imafdc-lp64d-lld/libc_pic.os.clean:(__start_context) > > There is a GNU ld issue: https://sourceware.org/bugzilla/show_bug.cgi?id=28509 > It incorrectly allows non-PLT non-GOT relocation referencing a preemptible symbol. > > > LLD uses consistent categories (e.g. R_PC / R_PLT_PC) to describe relocation types. > It has consistent diagnostic across all ports it supports. > GNU ld uses dispatches for each relocation type and may miss some diagnostics. > >> collect2: error: ld returned 1 exit status Indeed these are the same ARM issues, we will need to use internal symbols.