From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by sourceware.org (Postfix) with ESMTPS id 35C233858401 for ; Sun, 7 Nov 2021 14:24:46 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 35C233858401 Received: from pps.filterd (m0098410.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1A78I2a9005888; Sun, 7 Nov 2021 14:24:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3c6a93nb7k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 07 Nov 2021 14:24:44 +0000 Received: from m0098410.ppops.net (m0098410.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 1A7EOh3w006997; Sun, 7 Nov 2021 14:24:43 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 3c6a93nb7e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 07 Nov 2021 14:24:43 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 1A7ECRaB028992; Sun, 7 Nov 2021 14:24:42 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma01dal.us.ibm.com with ESMTP id 3c5hba8jht-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 07 Nov 2021 14:24:42 +0000 Received: from b01ledav006.gho.pok.ibm.com (b01ledav006.gho.pok.ibm.com [9.57.199.111]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 1A7EOetb52494806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 7 Nov 2021 14:24:40 GMT Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D61CBAC05E; Sun, 7 Nov 2021 14:24:40 +0000 (GMT) Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F5D0AC05B; Sun, 7 Nov 2021 14:24:40 +0000 (GMT) Received: from [9.211.99.43] (unknown [9.211.99.43]) by b01ledav006.gho.pok.ibm.com (Postfix) with ESMTP; Sun, 7 Nov 2021 14:24:40 +0000 (GMT) Message-ID: Date: Sun, 7 Nov 2021 08:24:39 -0600 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.3.0 Reply-To: wschmidt@linux.ibm.com Subject: Re: lld status with powerpc64 To: Adhemerval Zanella , Fangrui Song , Tulio Magno Quites Machado Filho , Alan Modra , Nemanja Ivanovic Cc: libc-alpha@sourceware.org References: <20211026200346.3371750-1-adhemerval.zanella@linaro.org> <20211026203327.6b2o5k4cmkuzzm6j@google.com> <87lf2ejdnw.fsf@linux.ibm.com> <87ilxijd5f.fsf@linux.ibm.com> <20211105072312.wbqrhgyjvsypkj52@google.com> <35b483cc-fb4e-6770-bff2-aebb5f178614@linaro.org> From: Bill Schmidt In-Reply-To: <35b483cc-fb4e-6770-bff2-aebb5f178614@linaro.org> Content-Type: text/plain; charset=UTF-8 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: jrVR9ulqMuzhaKwCKfdQXvue-B_NHQp8 X-Proofpoint-GUID: rmM5TyoYS7lTNmtho9L5d7paeXxusUFy Content-Transfer-Encoding: 8bit X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-11-07_08,2021-11-03_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 suspectscore=0 adultscore=0 clxscore=1011 mlxscore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 spamscore=0 mlxlogscore=999 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000 definitions=main-2111070090 X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, BODY_8BITS, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_EF, KAM_INFOUSMEBIZ, KAM_NUMSUBJECT, NICE_REPLY_A, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Nov 2021 14:24:50 -0000 Please coordinate with Alan Modra and Nemanja Ivanovic on this topic.  There are ongoing discussions about bfd and lld linker support around @notoc that will be resolved soon, and Tulio is on vacation, so I don't want the community to make steps they'll have to undo later, or for people to engage in duplicate work. Thanks! Bill On 11/5/21 8:58 AM, Adhemerval Zanella via Libc-alpha wrote: > > On 05/11/2021 04:23, Fangrui Song wrote: >> On 2021-10-27, Tulio Magno Quites Machado Filho wrote: >>> Tulio Magno Quites Machado Filho via Libc-alpha writes: >>> >>>> Fangrui Song via Libc-alpha writes: >>>> >>>>> On 2021-10-26, Adhemerval Zanella wrote: >>>>>>  #2  0x00007ffff7d7ae90 in __libc_start_main_impl () >>>>>>     from >>>>>>  /home/azanella/glibc/build/powerpc64le-linux-gnu-power9-lld/libc.so.6 >>>>>>  #3  0x0000000000000000 in ?? () >>>>>>  (gdb) disas >>>>>>  Dump of assembler code for function __gep_setup___vmx__sigjmp_save: >>>>>>  => 0x00007ffff7f2a980 <+0>:     .long 0x613ffe6 >>>>>>     0x00007ffff7f2a984 <+4>:     li      r12,-1280 >>>> This is a pla, but this GDB isn't able to disassemble it.  This instruction >>>> shouldn't be used unless when configuring glibc using --with-cpu=power10. >>> I can reproduce this issue even when configuring glibc with >>> --with-cpu=power9 --disable-multi-arch, which means the build should not have >>> any Power10 instructions. >>> >>> --  >>> Tulio Magno >> __gep_setup___vmx__sigjmp_save means a r12 setup stub in ld.lld and is >> used with R_PPC64_REL24_NOTOC for a non-PLT branch. >> >> Clang only emits `bl foo@notoc` with -mcpu=power10. >> However, sysdeps/powerpc/powerpc64/configure.ac enables USE_PPC64_NOTOC >> when the assembler (gas) supports it. >> >> >> Adhemerval noticed that ld.lld has a behavior difference from GNU ld: >> ld.lld defaults to PC-relative paddi stub while GNU ld doesn't (like >> --power10-stubs=no). >> Is USE_PPC64_NOTOC supposed to be used when targeting POWER9 and below? >> If yes, the ld.lld default should be fixed. > I figure out the issue and both bfd, gold, and lld align themselves on the > power10-stubs options handling. Using --power10-stubs without an arg is > equivalent to --power10-stubs=yes, but not specifying --power10-stubs at > all should be equivalent to --power10-stubs=auto. It somewhat confusing, > but I think it is to allow linker and compiler to be independently > regarding power10 stub generation. > > The issue is bfd enables power10 relocs generation on stubs iff it sees > the new pc-relative relocations (for instance R_PPC64_D34), otherwise > it generates default stubs (ppc64_elf_check_relocs:4700). > > So we have two options here: > > 1. Do not define USE_PPC64_NOTOC if with-lld (not optimal if ldd > aims to support such behavior). > > 2. Define USE_PPC64_NOTOC iff linker supports such optimization. > It means to emit a NOTOC relocation (R_PPC64_REL24_NOTOC), > link a simple binary without any pcrel and check if the stub has > power10 instruction. > > 3. Remove the USE_PPC64_NOTOC usage. It is used on setjmp routines > and on the syscall definition to call the __syscall_error. > > I am aiming to implement 2. since at least by disabling USE_PPC64_NOTOC > manually on config.h when configuring with lld I can build glibc. > >> >> In my testing environment (POWER9), ld.bfd doesn't support @notoc, so >> USE_PPC64_NOTOC is undefined. >> >> % gcc --version >> gcc (Debian 8.3.0-6) 8.3.0 >> ... >> % ld.bfd --version >> GNU ld (GNU Binutils for Debian) 2.31.1 >> ... >> >> I only see 8 more FAILs with ld.lld than ld.bfd > I am still having trouble to *finish* make check with lld release 13 > my environment (some tests stuck on infinite loop). > >> >> % diff -u 0 1 >> --- 0   2021-11-05 00:11:43.218731302 -0700 >> +++ 1   2021-11-05 00:11:37.659286448 -0700 >> @@ -9,6 +9,14 @@ >>  FAIL: debug/tst-lfschk6 >>  FAIL: dlfcn/bug-atexit3 >>  FAIL: elf/check-abi-libc >> +FAIL: elf/ifuncmain1pic >> +FAIL: elf/ifuncmain1pie >> +FAIL: elf/ifuncmain1vis >> +FAIL: elf/ifuncmain1vispic >> +FAIL: elf/ifuncmain1vispie >> +FAIL: elf/ifuncmain3 >> +FAIL: elf/ifuncmain6pie >> +FAIL: elf/tst-tlsopt-powerpc >>  FAIL: nptl/tst-cancel24 >>  FAIL: nptl/tst-minstack-throw >>  FAIL: nptl/tst-once5 >> >> >> I suspect ifuncmain1* is again related to the order of R_*_IRELATIVE with >> regard to R_*_JUMP_SLOT referencing a STT_GNU_IFUNC symbol. >> (something like https://maskray.me/blog/2021-01-18-gnu-indirect-function#relocation-resolving-order) >> But perhaps Adhemerval can look a look at it. > I still need to understand why armhf also fails with the ifunc using > protected symbols, although I am not if they are related to IRELATIVE > ordering. > >> >> For elf/tst-tlsopt-powerpc, it is simply because ld.lld doesn't implement the GNU ld powerpc64's >> __tls_get_addr_opt (pseudo-TLSDESC): https://maskray.me/blog/2021-02-14-all-about-thread-local-storage#powerpc-__tls_get_addr_opt >> Let me send a configure patch to disable it... >> >> Actually I do not know how to disable tst-tlsopt-powerpc properly. >> >> Perhaps add sysdeps/powerpc/configure.ac and move the >> sysdeps/unix/sysv/linux/powerpc/configure.ac --no-tls-get-addr-optimize >> to sysdeps/powerpc/configure.ac? >> sysdeps/powerpc/preconfigure.ac exists (I don't know how it is used). >> >> The patch requires some non-trivial configure.ac change, so I'd hope >> that an expert can do it > Maybe just disable the test if __tls_get_addr_opt (pseudo-TLSDESC) is not supported > by the linker? > > >> >> Hey, so lld linked glibc powerpc64 is in a pretty good status! > I am still not sure about it, I did could run some tests but I am still > struggling to get a make check to finish.