From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by sourceware.org (Postfix) with ESMTPS id 5D7523858CDB for ; Fri, 12 May 2023 22:47:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5D7523858CDB Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=dabbelt.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=dabbelt.com Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-643aad3bc41so9712736b3a.0 for ; Fri, 12 May 2023 15:47:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dabbelt-com.20221208.gappssmtp.com; s=20221208; t=1683931657; x=1686523657; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:from:to:cc:subject:date:message-id :reply-to; bh=J2ENMgAivbwcm38na6EM+bWE/hoahZK6RYqdV1plbXA=; b=0hqqiqZM/78oN4sGrSFXWsyg6e4JqIouQSuqZYLCoVhmlU8rsv844ZljQD2rrp+8F2 F4IhO09vmdPe5BLamPsVhL/IS7/qdldxsxBe94qFjKPR6AOiXI9rTdeZwgrZRdIJ7a7Z KiK+bM6t6hl5yOTD5j5NMl4EC1tTlLxK4fk5BkyZaKemUoa9N90yoq2XM1xNYTES4lDg 83vIgPjycD3FlXldT+pOV5hRFCW5pVH82F1SnWKYKz0Yr4euxLcwh6fgoYn2FHSRBWz5 kRCfMU8f+xAoewMrtz/25TWOP5Ru4JihXYKTwgvhCR9zsYerg2D7jmJUHIUNqu50IpoM EMyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683931657; x=1686523657; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=J2ENMgAivbwcm38na6EM+bWE/hoahZK6RYqdV1plbXA=; b=QSlR6eQa+zlm5rfqSHjLKLjO0r/k8dtQSEsWNcdFyFPAF0FcySNl//DrUZP7HEr7NC hUrgL0zUSu78imX10W0EeKoXwZAirGEb5xadOBTo8iWvUIitz/qWp+BghvbNFd1aveny y7Drg7MChAYLY5jsgxeA/WFGQdYzhYpH/8pucOG/dwH2N6dybQXglSawOsHkgWedVIFb pffAw8f0h7oflbZNoPpqnXoQqPYHzyAaCDomUus3htMQNY7SEvWRSHoL5I6Dx42WU9tL 4d6hyMzFb4I54Y6KaNTKhRK2fWPLUtTHZ0a6kQ52cJlrIbcQE/FHfkdVdndYcesR/Mvd /IhA== X-Gm-Message-State: AC+VfDyTNiHDLFaf15sh4Sf/L+LJpqd6Q0Kc0YPrwy7kxr+InR5dUMtA P/nhKw1l3ntvRqAbXiBZWCBj0w== X-Google-Smtp-Source: ACHHUZ4VDB6U3XIjVdn5esT8CEpCV2uJRZufgOiGGp7FBgNMZ/9taQSBzeWNSC7N6HEsnUKq33odtQ== X-Received: by 2002:a05:6a21:100e:b0:f5:7035:9a05 with SMTP id nk14-20020a056a21100e00b000f570359a05mr29022355pzb.29.1683931657237; Fri, 12 May 2023 15:47:37 -0700 (PDT) Received: from localhost ([135.180.227.0]) by smtp.gmail.com with ESMTPSA id j17-20020aa79291000000b0064378c52398sm7564131pfa.25.2023.05.12.15.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 May 2023 15:47:36 -0700 (PDT) Date: Fri, 12 May 2023 15:47:36 -0700 (PDT) X-Google-Original-Date: Fri, 12 May 2023 15:47:33 PDT (-0700) Subject: Re: global pointer gets overwritten with dlopen(3) on RISC-V In-Reply-To: <20230512223421.zvvbi374qx6kazel@google.com> CC: fweimer@redhat.com, l.stelmach@samsung.com, libc-alpha@sourceware.org, schwab@suse.de, adhemerval.zanella@linaro.org, joseph@codesourcery.com, binutils@sourceware.org, m.pikula@partner.samsung.com, m.szyprowski@samsung.com, k.lewandowsk@samsung.com, jeffreyalaw@gmail.com From: Palmer Dabbelt To: maskray@google.com Message-ID: Mime-Version: 1.0 (MHng) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,KAM_INFOUSMEBIZ,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Fri, 12 May 2023 15:34:21 PDT (-0700), maskray@google.com wrote: > On 2023-05-12, Palmer Dabbelt wrote: >>On Fri, 12 May 2023 14:09:08 PDT (-0700), maskray@google.com wrote: >>>On 2023-05-12, Palmer Dabbelt wrote: >>>>On Fri, 12 May 2023 13:11:43 PDT (-0700), fweimer@redhat.com wrote: >>>>>* Fangrui Song: >>>>> >>>>>>>1. Make -mno-relax the default for ld(1) (on Linux?). We have no >>>>>>>benchmarks whatsoever, but global variables aren't very popular in >>>>>>>application code these days and the gp register allows access to a >>>>>>>single memory page (4kB) only. No big deal really. >>>>>> >>>>>>I do agree that --no-relax-gp is a sensible default choice for GNU ld. >>>>>>https://maskray.me/blog/2021-03-14-the-dark-side-of-riscv-linker-relaxation#global-pointer-relaxation >>>>>> >>>>>>Perhaps you can start a separate topic on binutils? :) >>>>>> >>>>>>According to a doc from SiFive about -static -mcpu=sifive-u74 builds, >>>>>>https://docs.google.com/spreadsheets/d/14V7cPbyc80AcGHzsMaw9hYb232dzRbGCmTApnxj-SpU/edit#gid=0 >>>>>>global pointer relaxation saves at best 0.5% size (I guess that refers >>>>>>to .text. If we count all allocable sections, the percentage is likely >>>>>>even smaller.) >>>>> >>>>>For a mature toolchain, 0.5% in code size reduction would be *a lot*, >>>>>so I wouldn't dismiss that. >>>> >>>>That's broadly speaking why it sticks around. We've got a bunch of >>>>headaches related to relaxation, GP or otherwise, but they improve >>>>performance and nobody's figured out how to replace that yet. >>>> >>>>>Do we have a reproducer? Is the issue actually about gp relaxation for >>>>>the main executable? >>>> >>>>In general we don't reference GP from shared libraries as we don't >>>>have a GP save/restore scheme. There may be a bug floating around >>>>here somewhere, in which case we should fix it, but the original post >>>>sounds like it wasn't a supported use case. >>>> >>>>>Thanks, >>>>>Florian >>> >>>Global pointer relaxation only applies to +-2KiB data relative to __global_pointer$ (= .sdata + 0x800). >>>The area that potentially benefits global pointer relaxation is very small. >>> >>>0.5% code size reduction (relative to .text?) is the best case. I >>>suspect the program somehow has a lot global variable accesses and >>>placing these variables in .sdata helps. >>> >>>I've got results from Yingwei Zheng at PLCT lab using many >>>configurations. The saving is like 0.1%. >>>https://docs.google.com/spreadsheets/d/1Gz0h-C4U0toa9qELFtRaEWT_CzauE5JD9xMsLR8RyK8/edit#gid=1721258109 >>> >>>On the binutils side, we occasionally see patches to fix global pointer >>>relaxation bugs, e.g. the patch just sent a few hours ago: >>>https://sourceware.org/pipermail/binutils/2023-May/127413.html >>> >>>I do not know the embedded toolchain well, but for Linux desktop/server, >>>disabling global pointer relaxation seems like a sensible choice. If we >>>discover a better way to utilize GP (x3) in the future, disabling global >>>pointer relaxation today will result in fewer compatibility issues. >> >>This comes up all the time, you're just pushing for a backdoor ABI >>break. I get the desire to remove GP, if we were to be able to redo >>things I'd also not have it, but it's in the ABI and we can't change >>the binaries that exist. >> >>If you want a GP-free ABI then you should just go write one up. Then >>it'll become a distro problem, and if it turns out that users also >>don't want in then the GP ABI will rot and we can eventually deprecate >>it. > > I am advocating for a change in GNU ld to make --no-relax-gp the default > option, but I am not sure it can be considered an ABI break. > > When using ld --no-relax-gp, the conversion of code sequences to gp is > disabled, thus eliminating an assumption related to global pointer relaxation. > If an executable is relinked without global pointer relaxation, it should still > function properly. > > To the best of my knowledge, there is no official documentation that designates > linker relaxation as a mandatory feature. Relaxation schemes, including global > pointer relaxation, are optional. Making an optional feature opt-in does not > constitute an ABI break. > > glibc continues to initialize gp to __global_pointer$ to accommodate users who > opt-in for global pointer relaxation. Removing the initialization of gp would > indeed be an ABI break, and I have never proposed such a change. The ABI break is using GP for something else, but that always ends up being part of the argument for disabling GP-based relaxation. Without that the argument ends up just being that the GP-relaxations are too complicated for the benefit they provide. I certainly understand that argument, these (and for the rest of the relaxation stuff) is a huge amount of pain for a small amount of benefit (which seems to be mostly based on benchmark dragracing, which is never that strong of an argument). That said, last time we had this discussion was only a few months ago. I don't think anything has changed since then so I doubt things well go any differently. Just turning GP-based relaxation off by default doesn't get rid of the complexity, they're still in the ABI and people will still use them (even if just for benchmark dragracing) so they'll still need to work. > >>>Haiku >>>(https://github.com/riscv-non-isa/riscv-elf-psabi-doc/issues/298#issuecomment-1344724796), Android, and Fuchsia >>>have mentioned that they don't use global pointer relaxation.