From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by sourceware.org (Postfix) with ESMTPS id 31CA2383303D for ; Sun, 2 May 2021 15:27:39 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 31CA2383303D Received: by mail-ed1-f48.google.com with SMTP id d14so3482081edc.12 for ; Sun, 02 May 2021 08:27:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2Yo5lJFkseirK7+Os1dsvmENxr6gMP+2YIPJxFmkXVk=; b=MtMchnp9/8HgDfFOFzX2x/60v7lkyyKtXqKH56zPUB2z1Q9EmFomtqaD/fcjTOrFgV 8vTXFDUWarsKfty1Sge9hJK2YQWYmb5RwDCaCETMUohs9eAiQ9tFTrxTekJj1QeQMyDv +pHP5jDGE4SPZQ7RBBr/el2pKyPcAsPOxAlX4l6uKdDu7bJQ3KrfqDudGvxQ7xBLznfQ EapP9o4RlnQ2i1TZVH5m+V8U1KauBGfm0ICrpi8GBjb65lBzwikG+vm0Sedu3JM39Xb6 zEJIoW6Hwb6USrO3Hastarki9Nj7DQoV+shM1sIV2YCZD2gE/nox4RP1CoVecddxe87z 3kFw== X-Gm-Message-State: AOAM5322EVXAXYBgsixeoCEaLD+2nqkfdy1AwUpHFojsjTyN/Ydr2CM/ hJOKit9O7kTtvxtd7XRdH+gwKpW6BMF0nUD/BKg= X-Google-Smtp-Source: ABdhPJwhw4g3fWATRq9485rhmPqE20zAbxqrCkrtzYFFdBLT740uWpW4bDZ99Kccx/O2alF9MiVOiJ9uI1QA/dCpUgg= X-Received: by 2002:a05:6402:3079:: with SMTP id bs25mr16173762edb.146.1619969258275; Sun, 02 May 2021 08:27:38 -0700 (PDT) MIME-Version: 1.0 References: <20210415044258.GA6318@zn.tnic> <20210415052938.GA2325@1wt.eu> <20210415054713.GB6318@zn.tnic> <20210419141454.GE9093@zn.tnic> <20210419191539.GH9093@zn.tnic> <20210419215809.GJ9093@zn.tnic> In-Reply-To: From: Len Brown Date: Sun, 2 May 2021 11:27:26 -0400 Message-ID: Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features To: Borislav Petkov Cc: Willy Tarreau , Andy Lutomirski , Florian Weimer , "Bae, Chang Seok" , Dave Hansen , X86 ML , LKML , linux-abi@vger.kernel.org, "libc-alpha@sourceware.org" , Rich Felker , Kyle Huey , Keno Fischer Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 May 2021 15:27:40 -0000 On Fri, Apr 23, 2021 at 3:58 PM Borislav Petkov wrote: > > On Fri, Apr 23, 2021 at 03:35:30PM -0400, Len Brown wrote: > > Yes. If a library decides to execute AMX instructions on behalf > > of a task, the kernel will allocate an 8KB context switch buffer > > on behalf of that task. > > Again, the library should ask the kernel first whether it supports AMX. > And the process should decide whether to use AMX - not the library on > its own, on behalf of the process. > > > Granted, if you find a reason to dislike AMX, the mechanisms to disable > > it today are on a system-wide basis, not on a process or task basis. > > Again, I don't dislike the feature. I don't want libraries jumping on > new features without asking the process or the kernel first especially > when those features have performance implications and need kernel > support. Here is how it works: 1. The kernel boots and sees the feature in CPUID. 2. If the kernel supports that feature, it sets XCR0[feature]. For some features, there may be a bunch of kernel support, while simple features may require only state save/restore. 2a. If the kernel doesn't support the feature, XCR0[feature] remains cleared. 3. user-space sees the feature in CPUID 4. user-space sees for the feature via xgetbv[XCR0] 5. If the feature is enabled in XCR0, the user happily uses it. For AMX, Linux implements "transparent first use" so that it doesn't have to allocate 8KB context switch buffers for tasks that don't actually use AMX. It does this by arming XFD for all tasks, and taking a #NM to allocate a context switch buffer only for those tasks that actually execute AMX instructions. 5a. If the feature is not enabled in XCR0, and the tasks uses those instructions anway, the hardware delivers a #UD and the kernel kills the process with a signal. So you already have what you want, WRT user-space being required to ask the kernel if the feature is supported. When the kernel sets XCR0[feature], it tells user-space that the kernel supports the feature; and there is no way that user space can use the feature if the kernel did not set that bit. System calls before (and after) using a feature are not necessary, and would only degrade performance over the existing ABI. Regarding performance implications... The system administrator is empowered to enable or disable a feature in BIOS (clears CPUID bit) or in the kernel (clears XCR0 bit) if they don't like it. Yes, this is a per-system decision, rather than a per-process or a per-thread decision. So the only thing you don't have that you asked for is a way for the main process to control what instructions are used by the libraries that it links with. That one is above my pay grade. Does the application have a *choice* of which libraries they link with? cheers, Len Brown, Intel Open Source Technology Center