From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-xe29.google.com (mail-vs1-xe29.google.com [IPv6:2607:f8b0:4864:20::e29]) by sourceware.org (Postfix) with ESMTPS id D5B63385AE4E for ; Wed, 16 Aug 2023 23:18:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D5B63385AE4E Authentication-Results: sourceware.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=google.com Received: by mail-vs1-xe29.google.com with SMTP id ada2fe7eead31-44ae69a6b20so561941137.1 for ; Wed, 16 Aug 2023 16:18:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1692227916; x=1692832716; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=FaRzVW71TW51uAYR+1giFH4ox4PAzo8/kqNo09DKa/4=; b=AplJ4MPBRZKEQeF4LxQMIeV0sdDY54dHkru+nprA123ckhe6oX/jIJ+5TiGPtIIEMZ 0dVgsZNNKUMcQV5hD7h3DpYMZcI2/yX+9nqbGeJdRBS8CG0fQtXkoeR3tiJC8BDilA2C IsuujpbFUKcRM+C61qqWWvaNCEjFOQA3vbQSVN0pZFgti0JhpXd2Amn6fzGwbN71xDXk ItjGO/Wc+xo4RScenHAuq9aJOB70uo/nX0tFJ3OaQaInJ3ZL7HqDbGZaVvkuiAKVmdcN Mwz5WgTacSXLPavodgvGjAUkKp4mRtFIe7T9CJI8KFRSnNYePlK/498l6yPmK/gHxzZn KdTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692227916; x=1692832716; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FaRzVW71TW51uAYR+1giFH4ox4PAzo8/kqNo09DKa/4=; b=GUnQifjqhDACVl/UKOqCowte9iR3lk+3gsOOBdk+nIjGf0QmZEmYPNGXEM0f1tMgzT aZhRyFdfxa6kxpvWalFL8EfsvoG3k3EVQV1JtMHHCQ+hLEW26xMoF6k2XLfpBfnpYiAm w9sMrDHnvlM+/w+/st4Bxy79+moLBDYjTVORatGmqx56o+7ZFZxAHBmrvVzdo/mRW6fJ v10QEwsGFF08JI/0WmlKGaQ8XJuDPyJyp4zrnciLM3g/3chV9M8NxVmE957eM1JrWoB3 2ZPWtQBW0QmIvERsTfUiAGyuWnaCu9zSE4LLBcsZprTk3FupeXLjQ6Ix1gYnioUbxJ/y 9ciA== X-Gm-Message-State: AOJu0YymSsCAYsL8eqN5qFRqoD37emMuA5K9lUZeztGpeAKixc2SPzCN zgAZ6x9P28rXQm8jddHfqZEvYNUi7wSxi3QG+sH1LA== X-Google-Smtp-Source: AGHT+IGbIb5l/dIejx1Dt0+Usa6UVdUz8lfOxymQNJqDGVuCE3OFLhnW2ztnW/vw9xTiaHkwF7b/iq8VXQJ5GtNCKdQ= X-Received: by 2002:a67:f859:0:b0:443:895d:1b53 with SMTP id b25-20020a67f859000000b00443895d1b53mr3063493vsp.10.1692227915887; Wed, 16 Aug 2023 16:18:35 -0700 (PDT) MIME-Version: 1.0 References: <20230802155903.2552780-1-evan@rivosinc.com> <20230802155903.2552780-6-evan@rivosinc.com> <87il9w37vi.fsf@oldenburg.str.redhat.com> <6f0911c6-b24b-444c-4b4b-a62e49a51734@linaro.org> <548fc7d5-6225-69e7-f4a7-47669d2fdbd5@linaro.org> In-Reply-To: From: enh Date: Wed, 16 Aug 2023 16:18:24 -0700 Message-ID: Subject: Re: [PATCH v6 5/5] riscv: Add and use alignment-ignorant memcpy To: Evan Green Cc: Richard Henderson , Florian Weimer , libc-alpha@sourceware.org, slewis@rivosinc.com, palmer@rivosinc.com, vineetg@rivosinc.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-18.1 required=5.0 tests=BAYES_00,DKIMWL_WL_MED,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, Aug 15, 2023 at 4:02=E2=80=AFPM Evan Green wrot= e: > > On Tue, Aug 15, 2023 at 2:54=E2=80=AFPM enh wrote: > > > > On Tue, Aug 15, 2023 at 9:41=E2=80=AFAM Evan Green = wrote: > > > > > > On Fri, Aug 11, 2023 at 5:01=E2=80=AFPM enh wrote: > > > > > > > > On Mon, Aug 7, 2023 at 5:01=E2=80=AFPM Evan Green wrote: > > > > > > > > > > On Mon, Aug 7, 2023 at 3:48=E2=80=AFPM enh wrote= : > > > > > > > > > > > > On Mon, Aug 7, 2023 at 3:11=E2=80=AFPM Evan Green wrote: > > > > > > > > > > > > > > On Thu, Aug 3, 2023 at 3:30=E2=80=AFPM Richard Henderson > > > > > > > wrote: > > > > > > > > > > > > > > > > On 8/3/23 11:42, Evan Green wrote: > > > > > > > > > On Thu, Aug 3, 2023 at 10:50=E2=80=AFAM Richard Henderson > > > > > > > > > wrote: > > > > > > > > >> Outside libc something is required. > > > > > > > > >> > > > > > > > > >> An extra parameter to ifunc is surprising though, and cl= early not ideal per the extra > > > > > > > > >> hoops above. I would hope for something with hidden vis= ibility in libc_nonshared.a that > > > > > > > > >> could always be called directly. > > > > > > > > > > > > > > > > > > My previous spin took that approach, defining a > > > > > > > > > __riscv_hwprobe_early() in libc_nonshared that could rout= e to the real > > > > > > > > > function if available, or make the syscall directly if no= t. But that > > > > > > > > > approach had the drawback that ifunc users couldn't take = advantage of > > > > > > > > > the vDSO, and then all users had to comprehend the differ= ence between > > > > > > > > > __riscv_hwprobe() and __riscv_hwprobe_early(). > > > > > > > > > > > > > > > > I would define __riscv_hwprobe such that it could take adva= ntage of the vDSO once > > > > > > > > initialization reaches a certain point, but cope with being= run earlier than that point by > > > > > > > > falling back to the syscall. > > > > > > > > > > > > > > > > That constrains the implementation, I guess, in that it can= 't set errno, but just > > > > > > > > returning the negative errno from the syscall seems fine. > > > > > > > > > > > > > > > > It might be tricky to get a reference to GLRO(dl_vdso_riscv= _hwprobe) very early, but I > > > > > > > > would hope that some application of __attribute__((weak)) m= ight correctly get you a NULL > > > > > > > > prior to full relocations being complete. > > > > > > > > > > > > > > Right, this is what we had in the previous iteration of this = series, > > > > > > > and it did work ok. But it wasn't as good since it meant ifun= c > > > > > > > selectors always got stuck in the null/fallback case and were= forced > > > > > > > to make the syscall. With this mechanism they get to take adv= antage of > > > > > > > the vDSO. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In contrast, IMO this approach is much nicer. Ifunc write= rs are > > > > > > > > > already used to getting hwcap info via a parameter. Addin= g this second > > > > > > > > > parameter, which also provides hwcap-like things, seems l= ike a natural > > > > > > > > > extension. I didn't quite follow what you meant by the "e= xtra hoops > > > > > > > > > above". > > > > > > > > > > > > > > > > The check for null function pointer, for sure. But also co= nsider how __riscv_hwprobe is > > > > > > > > going to be used. > > > > > > > > > > > > > > > > It might be worth defining some helper functions for probin= g a single key or a single > > > > > > > > field. E.g. > > > > > > > > > > > > > > > > uint64_t __riscv_hwprobe_one_key(int64_t key, unsigned int = flags) > > > > > > > > { > > > > > > > > struct riscv_hwprobe pair =3D { .key =3D key }; > > > > > > > > int err =3D __riscv_hwprobe(&pair, 1, 0, NULL, flags); > > > > > > > > if (err) > > > > > > > > return err; > > > > > > > > if (pair.key =3D=3D -1) > > > > > > > > return -ENOENT; > > > > > > > > return pair.value; > > > > > > > > } > > > > > > > > > > > > > > > > This implementation requires that no future hwprobe key def= ine a value which as a valid > > > > > > > > value in the errno range (or better, bit 63 unused). Alter= nately, or additionally: > > > > > > > > > > > > > > > > bool __riscv_hwprobe_one_mask(int64_t key, uint64_t mask, u= int64_t val, int flags) > > > > > > > > { > > > > > > > > struct riscv_hwprobe pair =3D { .key =3D key }; > > > > > > > > return (__riscv_hwprobe(&pair, 1, 0, NULL, flags) =3D=3D= 0 > > > > > > > > && pair.key !=3D -1 > > > > > > > > && (pair.value & mask) =3D=3D val); > > > > > > > > } > > > > > > > > > > > > > > > > These yield either > > > > > > > > > > > > > > > > int64_t v =3D __riscv_hwprobe_one_key(CPUPERF_0, 0); > > > > > > > > if (v >=3D 0 && (v & MISALIGNED_MASK) =3D=3D MISALIGNE= D_FAST) > > > > > > > > return __memcpy_noalignment; > > > > > > > > return __memcpy_generic; > > > > > > > > > > > > > > > > or > > > > > > > > > > > > > > > > if (__riscv_hwprobe_one_mask(CPUPERF_0, MISALIGNED_MAS= K, MISALIGNED_FAST, 0)) > > > > > > > > return __memcpy_noalignment; > > > > > > > > return __memcpy_generic; > > > > > > > > > > > > > > > > which to my mind looks much better for a pattern you'll be = replicating so very many times > > > > > > > > across all of the ifunc implementations in the system. > > > > > > > > > > > > > > Ah, I see. I could make a static inline function in the heade= r that > > > > > > > looks something like this (mangled by gmail, sorry): > > > > > > > > > > > > > > /* Helper function usable from ifunc selectors that probes a = single key. */ > > > > > > > static inline int __riscv_hwprobe_one(__riscv_hwprobe_t hwpro= be_func, > > > > > > > signed long long int key, > > > > > > > unsigned long long int *value) > > > > > > > { > > > > > > > struct riscv_hwprobe pair; > > > > > > > int rc; > > > > > > > > > > > > > > if (!hwprobe_func) > > > > > > > return -ENOSYS; > > > > > > > > > > > > > > pair.key =3D key; > > > > > > > rc =3D hwprobe_func(&pair, 1, 0, NULL, 0); > > > > > > > if (rc) { > > > > > > > return rc; > > > > > > > } > > > > > > > > > > > > > > if (pair.key < 0) { > > > > > > > return -ENOENT; > > > > > > > } > > > > > > > > > > > > > > *value =3D pair.value; > > > > > > > return 0; > > > > > > > } > > > > > > > > > > > > > > The ifunc selector would then be significantly cleaned up, lo= oking > > > > > > > something like: > > > > > > > > > > > > > > if (__riscv_hwprobe_one(hwprobe_func, RISCV_HWPROBE_KEY_CPUPE= RF_0, &value)) > > > > > > > return __memcpy_generic; > > > > > > > > > > > > > > if (value & RISCV_HWPROBE_MISALIGNED_MASK) =3D=3D RISCV_HWPRO= BE_MISALIGNED_FAST) > > > > > > > return __memcpy_noalignment; > > > > > > > > > > > > (Android's libc maintainer here, having joined the list just to= talk > > > > > > about risc-v ifuncs :-) ) > > > > > > > > > > > > has anyone thought about calling ifunc resolvers more like this= ... > > > > > > > > > > > > --same part of the dynamic loader that caches the two getauxval= ()s for arm64-- > > > > > > static struct riscv_hwprobe probes[] =3D { > > > > > > {.value =3D RISCV_HWPROBE_KEY_MVENDORID}, > > > > > > {.value =3D RISCV_HWPROBE_KEY_MARCHID}, > > > > > > {.value =3D RISCV_HWPROBE_KEY_MIMPID}, > > > > > > {.value =3D RISCV_HWPROBE_KEY_BASE_BEHAVIOR}, > > > > > > {.value =3D RISCV_HWPROBE_KEY_IMA_EXT}, > > > > > > {.value =3D RISCV_HWPROBE_KEY_CPUPERF_0}, > > > > > > ... // every time a new key is added to the kernel, we add it h= ere > > > > > > }; > > > > > > __riscv_hwprobe(...); // called once > > > > > > > > > > > > --part of the dynamic loader that calls ifunc resolvers-- > > > > > > (*ifunc_resolver)(sizeof(probes)/sizeof(probes[0]), probes); > > > > > > > > > > > > this is similar to what we already have for arm64 (where there'= s a > > > > > > getauxval(AT_HWCAP) and a pointer to a struct for AT_HWCAP2 and > > > > > > potentially others), but more uniform, and avoiding the source > > > > > > (in)compatibility issues of adding new fields to a struct [even= if it > > > > > > does have a size_t to "version" it like the arm64 ifunc struct]= . > > > > > > > > > > > > yes, it means everyone pays to get all the hwprobes, but that g= ets > > > > > > amortized. and lookup in the ifunc resolver is simple and quick= . if we > > > > > > know that the keys will be kept dense, we can even have code in= ifunc > > > > > > resolvers like > > > > > > > > > > > > if (probes[RISCV_HWPROBE_BASE_BEHAVIOR_IMA].value & RISCV_HWPRO= BE_IMA_V) ... > > > > > > > > > > > > though personally for the "big ticket items" that get a letter = to > > > > > > themselves like V, i'd be tempted to pass `(getauxval(AT_HWCAP)= , > > > > > > probe_count, probes_ptr)` to the resolver, but i hear that's > > > > > > controversial :-) > > > > > > > > > > Hello, welcome to the fun! :) > > > > > > > > (sorry for the delay. i've been thinking :-) ) > > > > > > > > > What you're describing here is almost exactly what we did inside = the > > > > > vDSO function. The vDSO function acts as a front for a handful of > > > > > probe values that we've already completed and cached in userspace= . We > > > > > opted to make it a function, rather than exposing the data itself= via > > > > > vDSO, so that we had future flexibility in what elements we cache= d in > > > > > userspace and their storage format. We can update the kernel as n= eeded > > > > > to cache the hottest things in userspace, even if that means > > > > > rearranging the data format, passing through some extra informati= on, > > > > > or adding an extra snip of code. My hope is callers can directly > > > > > interact with the vDSO function (though maybe as Richard suggeste= d > > > > > maybe with the help of a tidy inline helper), rather than trying = to > > > > > add a second layer of userspace caching. > > > > > > > > on reflection i think i might be too focused on the FMV use case, i= n > > > > part because we're looking at those compiler-generated ifuncs for > > > > arm64 on Android atm. i think i'm imagining a world where there's a > > > > lot of that, and worrying about having to pay for the setup, call, = and > > > > loop for each ifunc, and wondering why we don't just pay once inste= ad. > > > > (as a bit of background context, Android "app" start is actually a > > > > dlopen() in a clone of an existing zygote process, and in general a= pp > > > > launch time is one of the key metrics anyone who's serious is > > > > optimizing for. you'd be surprised how much of my life i spend > > > > explaining to people that if they want dlopen() to be faster, maybe > > > > they shouldn't ask us to run thousands of ELF constructors.) > > > > > > > > but... the more time i spend looking at what we actually need in > > > > third-party open source libraries right now i realize that libc and > > > > FMV (which is still a future thing for us anyway) are really the on= ly > > > > _actual_ ifunc users. perhaps in part because macOS/iOS don't have > > > > ifuncs, all the libraries that are part of the OS itself, for examp= le, > > > > are just doing their own thing with function pointers and > > > > pthread_once() or whatever. > > > > > > > > (i have yet to try to get any data on actual apps. i have no reason= to > > > > think they'll be very different, but that could easily be skewed by > > > > popular middleware or a popular game engine using ifuncs, so i do p= lan > > > > on following up on that.) > > > > > > > > "how do they decide what to set that function pointer to?". well, i= t > > > > looks like in most cases cpuid on x86 and calls to getauxval() > > > > everywhere else. in some cases that's actually via some other libra= ry: > > > > https://github.com/pytorch/cpuinfo or > > > > https://github.com/google/cpu_features for example. so they have a > > > > layer of caching there, even in cases where they don't have a singl= e > > > > function that sets all the function pointers. > > > > > > Right, function multi-versioning is just the sort of spot where we'd > > > imagine hwprobe gets used, since it's providing similar/equivalent > > > information to what cpuid does on x86. It may not be quite as fast as > > > cpuid (I don't know how fast cpuid actually is). But with the vDSO > > > function+data in userspace it should be able to match getauxval() in > > > performance, as they're both a function pointer plus a loop. We're > > > sort of planning for a world in which RISC-V has a wider set of these > > > values to fetch, such that a ifunc selector may need a more complex > > > set of information. Hwprobe and the vDSO gives us the ability both to > > > answer multiple queries fast, and freely allocate more keys that may > > > represent versioned features or even compound features. > > > > yeah, my incorrect mental model was that -- primarily because of > > x86-64 and cpuid -- every function would get its own ifunc resolver > > that would have to make a query. but the [in progress] arm64 > > implementation shows that that's not really the case anyway, and we > > can just cache __riscv_hwprobe() in the same [one] place that > > getauxval() is already being cached for arm64. > > Sounds good. > > > > > > > so assuming i don't find that apps look very different from the OS > > > > (that is: that apps use lots of ifuncs), i probably don't care at a= ll > > > > until we get to FMV. and i probably don't care for FMV, because > > > > compiler-rt (or gcc's equivalent) will be the "caching layer" there= . > > > > (and on Android it'll be a while before i have to worry about libc'= s > > > > ifuncs because we'll require V and not use ifuncs there for the > > > > foreseeable future.) > > > > > > > > so, yeah, given that i've adopted the "pass a null pointer rather t= han > > > > no arguments" convention you have, we have room for expansion if/wh= en > > > > FMV is a big thing, and until then -- unless i'm shocked by what i > > > > find looking at actual apps -- i don't think i have any reason to > > > > believe that ifuncs matter that much, and if compiler-rt makes one > > > > __riscv_hwprobe() call per .so, that's probably fine. (i already sp= end > > > > a big chunk of my life advising people to just have one .so file, > > > > exporting nothing but a JNI_OnLoad symbol, so this will just make t= hat > > > > advice even better advice :-) ) > > > > > > Just to confirm, by "pass a null pointer", you're saying that the > > > Android libc also passes NULL as the second ifunc selector argument > > > (or first)? > > > > #elif defined(__riscv) > > // This argument and its value is just a placeholder for now, > > // but it means that if we do pass something in future (such as > > // getauxval() and/or hwprobe key/value pairs), callees will be able = to > > // recognize what they're being given. > > typedef ElfW(Addr) (*ifunc_resolver_t)(void*); > > return reinterpret_cast(resolver_addr)(nullptr); > > > > it's arm64 that has the initial getauxval() argument: > > > > #if defined(__aarch64__) > > typedef ElfW(Addr) (*ifunc_resolver_t)(uint64_t, __ifunc_arg_t*); > > static __ifunc_arg_t arg; > > static bool initialized =3D false; > > if (!initialized) { > > initialized =3D true; > > arg._size =3D sizeof(__ifunc_arg_t); > > arg._hwcap =3D getauxval(AT_HWCAP); > > arg._hwcap2 =3D getauxval(AT_HWCAP2); > > } > > return reinterpret_cast(resolver_addr)(arg._hwcap > > | _IFUNC_ARG_HWCAP, &arg); > > > > https://android.googlesource.com/platform/bionic/+/main/libc/bionic/bio= nic_call_ifunc_resolver.cpp > > > > > That's good. It sounds like you're planning to just > > > continue passing NULL for now, and wait for people to start clamoring > > > for this in android libc? > > > > yeah, and i'm assuming there will never be any clamor ... yesterday > > and today i actually checked a bunch of popular apks, and didn't find > > any that were currently using ifuncs. > > > > the only change i'm thinking of making right now is that "there's a > > single argument, and it's null" should probably be the default. > > obviously since Android doesn't add new architectures very often, this > > is only likely to affect x86/x86-64 for the foreseeable future, but > > being able to recognize at a glance "am i running under a libc new > > enough to pass me arguments?" would certainly have helped for arm64. > > even if x86/x86-64 never benefit, it seems like the right default for > > the #else clause... > > Sounds good, thanks for the pointers. The paranoid person in me would > also add a comment in the risc-v section that if a pointer to hwprobe > is added, it should be added as the second argument, behind hwcap as > the first (assuming this change lands). > > Come to think of it, the static inline helper I'm proposing in my > discussion with Richard needs to take both arguments, since callers > need to check both ((arg1 !=3D 0) && (arg2 !=3D NULL)) to safely know tha= t > arg2 is a pointer to __riscv_hwprobe(). presumably not `(arg1 !=3D 0)` but `(arg1 & _IFUNC_ARG_HWCAP)` to match arm= 64? > -Evan