From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) by sourceware.org (Postfix) with ESMTPS id 4BC563834843 for ; Fri, 9 Dec 2022 19:58:52 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4BC563834843 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=sifive.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=sifive.com Received: by mail-qt1-x829.google.com with SMTP id s9so4167677qtx.6 for ; Fri, 09 Dec 2022 11:58:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ZAsqCIFgPt6BSntE2zK/1grKaxatSUFCPk0z0wIkAOI=; b=UE0ilwfG5ft6c+WNo48Hu5oJEUZt2YPjH/PVnZ40KzSKlmJi3jIAD3drUF6Tw1bJQN Q52cdO2w12c+G3TuI4zkOo20GjaJT9NksWJ8C1f8TW4yWn0TEvxnPC2Z0lQI7U1Gp6Rh bbRoIfjihl5wOovQ+clxwm7M/fvK8guNtDptvNEq29G+yWnjNeGmOtKD4b40VtEEQpcn t+UkhotjE+UELGPLOceW0Aln8BhFPJNwPpuw0lKZPrfEGORAGihSKvJGdYhftF2GJdEG M3LIUnLhFOvnwQJCUNnMkLuWhybgwLZq1W+dIoFcWJFWzSlM451nT8CyPgTNQT5Bwa1u 8A9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZAsqCIFgPt6BSntE2zK/1grKaxatSUFCPk0z0wIkAOI=; b=z1myfS3cyIqPw7bAW828lKk4J8Pipp42zPAf2+yUrayq0NPb9K+RmJl9bA0qwUc2Wa c7BR6vRyhY4MlPKUiRe5r/A+wL+pnfmzwp1nNldHPJvJP78rui2yKFSLyWC5uxAWdDZg o16J+tmMOAnye2mAQ/cRP4eAa+DnRJfbY9zxnoWeQG9zhA3cNP0Wa5j6z1MRl6Sjia7o fW7yA+58rtsFYUPFpUjkKpAmI/rgSP8z9zZdsCpI/eN6R27u8U6kPrtQm0wLO2zdzAE5 Aoxu5WUGz4yjbxdhI8sFjbQLMtYTorSRZvJgSwswOxl9lYXvjjONgqrBHOHaVNsmE8Mp aBLw== X-Gm-Message-State: ANoB5pkAOcCCxlm0PZtKaJP9Gr9Ay3ZpxJrVBd1wPtX+JdqiCTyjBnz6 3A/48FcMEVOd9OMlI32p4J98VcrGpYak9Co6w7aSy6o6IIsMhMKAnay7U3hxLv0NjloLwlaG1mJ XdbTnYYR4OWq06LkKreWuxebTD+qYrs7rt/S5zWpjpuPqh3VA8J6J7Qo7jkTHd0ysTCG5uF+AwA == X-Google-Smtp-Source: AA0mqf5xGHBMtcpvbEv7OLUN+6etMWbLuUP4bnkY2R9hVjORfHnDUtghN1pDgT6p4mU6PeAFkje7Kg== X-Received: by 2002:ac8:4b7a:0:b0:3a6:9621:e7d4 with SMTP id g26-20020ac84b7a000000b003a69621e7d4mr7918231qts.18.1670615930884; Fri, 09 Dec 2022 11:58:50 -0800 (PST) Received: from mail-yb1-f181.google.com (mail-yb1-f181.google.com. [209.85.219.181]) by smtp.gmail.com with ESMTPSA id 195-20020a370ccc000000b006fec1c0754csm496252qkm.87.2022.12.09.11.58.48 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 09 Dec 2022 11:58:49 -0800 (PST) Received: by mail-yb1-f181.google.com with SMTP id s11so6783403ybe.2 for ; Fri, 09 Dec 2022 11:58:48 -0800 (PST) X-Received: by 2002:a5b:b45:0:b0:6f8:ce8a:57a5 with SMTP id b5-20020a5b0b45000000b006f8ce8a57a5mr40756849ybr.26.1670615928410; Fri, 09 Dec 2022 11:58:48 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Andrew Waterman Date: Fri, 9 Dec 2022 11:58:37 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: RISCV Vector unit disabled by default for new task (was Re: [PATCH v12 17/17] riscv: prctl to enable vector commands) To: Vineet Gupta Cc: Palmer Dabbelt , fweimer@redhat.com, stillson@rivosinc.com, Paul Walmsley , anup@brainfault.org, atishp@atishpatra.org, guoren@kernel.org, Conor Dooley , greentime.hu@sifive.com, vincent.chen@sifive.com, andy.chiu@sifive.com, arnd@kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, bjorn@kernel.org, libc-alpha@sourceware.org, christoph.muellner@vrull.eu, Aaron Durbin , linux@rivosinc.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Fri, Dec 9, 2022 at 11:42 AM Vineet Gupta wrote: > > > On 12/9/22 09:21, Palmer Dabbelt wrote: > > On Fri, 09 Dec 2022 05:04:23 PST (-0800), fweimer@redhat.com wrote: > >> * Darius Rad: > >> > >>> On Fri, Dec 09, 2022 at 01:32:33PM +0100, Florian Weimer via > >>> Libc-alpha wrote: > >>>> * Darius Rad: > >>>> > >>>> > On Fri, Dec 09, 2022 at 11:02:57AM +0100, Florian Weimer wrote: > >>>> >> * Andrew Waterman: > >>>> >> > >>>> >> > This suggests that ld.so, early-stage libc, or possibly both > >>>> will need > >>>> >> > to make this prctl() call, perhaps by parsing the ELF headers > >>>> of the > >>>> >> > binary and each library to determine if the V extension is used. > >>>> >> > >>>> >> If the string functions use the V extension, it will be enabled > >>>> >> unconditionally. So I don't see why it's okay for libc to > >>>> trigger this > >>>> >> alleged UAPI change, when the kernel can't do it by default. > >>>> >> > >>>> > > >>>> > Because the call to enable can fail and userspace needs to deal > >>>> with that. > >>>> > >>>> Failure is usually indicated by an AT_HWCAP or AT_HWCAP2 bit remaining > >>>> zero, or perhaps a special CPU register (although that is more > >>>> unusual). > >>> > >>> That would indicate that the extension is not present, which is one > >>> of, but > >>> not the only way it can fail. > >> > >> I think you should bring down the number of failure modes. HWCAP has > >> the advantage that it communicates kernel/hypervisor/firmware/CPU > >> support in a single bit, which simplifies the programming model and > >> avoids hard-to-detect bugs. It's not clear why it would be beneficial > >> to continue on ENOMEM failures here because the system must clearly be > >> in bad shape at this point, and launching a new process is very unlikely > >> to improve matters. So I think the simpler programming model is the way > >> to go here. > >> > >>> The vector extension relies on dynamically allocated memory in the > >>> kernel, > >>> which can fail. > > > > The issue I'm worried about is that V needs more space in the > > ucontext-type structures. We have an extensibility scheme there so we > > think it can be made to work, but IIUC we'll need glibc to be updated > > to handle the extended contexts in order to avoid losing state when > > doing ucontext-related operations like signal handling. > > Sorry this is not relevant to this thread. I started a different thread > on ABI/sigcontext aspects, lets discuss it there. > > > > > I don't see a way to handle that with just HWCAP, as we essentially > > need some bi-directional communicaton between userspace and the kernel > > so they can both decide to turn on V. I don't think we strictly need > > a system call to do that, we kicked around the idea of encoding this > > in the ELF, but there's a lot of flavors of vector in RISC-V and we've > > had trouble trying to encode these in binaries before so it seems > > easier to just use the syscall. > > > >> But this failure can be reported as part of execve and clone. > >> > >>> It also provides the opportunity for the kernel to deny access to the > >>> vector extension, perhaps due to administrative policy or other future > >>> mechanism. > >> > >> HWCAP can do this, too. > > Having the prctl as general purpose knob to disable the V unit for > various reasons makes sense. > > But keeping the V unit disabled by default and using prctl as a > gatekeeper to enable it feels unnecessary and tedious. > Here's my reasoning below (I'm collating comments from prior msgs as well). > > 1. Doesn't it add another userspace ABI which is already a headache for > this feature. And that needs to be built into not just libc but > potentially other runtimes too. Even after implemention there will be an > interim pain as the new prctl takes time to trickle down into tooling > and headers. Besides the new stuff will never be compatible with older > kernel but that is a minor point since older kernel not supporting V is > a deal breaker anyways. > > 2. People want the prctl gatekeeping for ability to gracefully handle > memory allocation failure for the extra V-state within kernel. But that > is only additional 4K (for typical 128 wide V regs) per task. If that is > failing, the system is not doing well anyways. Besides it is not an > issue at all since ENOMEM in clone/execve for the additional space > should handle the failure anyways. Only very sophisticated apps would > downgrade from executing V to Scalar code if the prctl failed. Instead > memory allocation is more likely to be an issue when copying V state on > a deep user stack across signal handling but there's nothing we can do > about it. > > 3. Another argument to prctl gatekeeping is ensuring user-space conforms > to vector length. But isn't the holy grail of RV V-extension VLA (Vector > Length Agnostic) programming. Yes, a suitable ABI for the V extension should cater cleanly to the VLA model, since that's a major selling point of this ISA. The baseline assumption should be that programs will execute correctly regardless of VLEN (subject to the constraint that the V extension requires VLEN >= 128, of course). It's of course valid to construct programs with VLEN-dependent behavior (e.g. dynamic dispatch to routines optimized differently for different VLEN), but it should be considered the program's responsibility to get that right. I don't think the ABI needs to furnish guard rails. > I expect most implements to follow that. > If there are super sophisticated (or dumb) apps that don't follow, they > will fail randomly. I think of Vector Length as any other ISA extensions > - its not that currently apps are required to prctl() for bitmanip > extension if they want to use it. Sure they could use AT_HWCAP (or > /proc/cpuinfo or any other portable way) to query the capability, same > can be done for V as well. Besides vlen is readable from user space so > the app can read it to make sure. My worry is we are providing > additional safety net to a small category of apps at the expense of > making it tiresome for everyone else. > > HWCAP solves the kernel to user-space communication of capabilities. The > prctl is for user-space to kernel communication but I'm not convinced it > is really solving problems for the common case. > > Thx, > -Vineet