From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 64A003858D28 for ; Mon, 17 Jul 2023 08:38:58 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 64A003858D28 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1689583137; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=kaXs3M3bwOW5qK4t4esxqK02hzuChm6YuEhSl5P/JTg=; b=i/N5Y6eR0NExoxUUcB5yhPzawqVgIQAqEgmF+igAahuceVuUb300OWeBAYGo26NccSfmCu DPcmBJRn7Nxwet/1WJoi0rYTm0Kv6GJF+aCTLhfRoZkKpEA13iSN9FrDU7mckia9xyCqtU WpWvqJvl1gdxkIL/n7C42nVmWJtBBD4= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-264-paWaAmuIOheGNfLMT-huXw-1; Mon, 17 Jul 2023 04:38:54 -0400 X-MC-Unique: paWaAmuIOheGNfLMT-huXw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F3103101A54E; Mon, 17 Jul 2023 08:38:53 +0000 (UTC) Received: from oldenburg.str.redhat.com (unknown [10.2.16.3]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C775B40C2063; Mon, 17 Jul 2023 08:38:52 +0000 (UTC) From: Florian Weimer To: libc-alpha@sourceware.org, qemu-devel@nongnu.org Cc: "H. Peter Anvin" , Vitaly Kuznetsov Subject: Missing cache information on x86-64 under Intel TDX (glibc bug 30643) Date: Mon, 17 Jul 2023 10:38:50 +0200 Message-ID: <87mszv7x0l.fsf@oldenburg.str.redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: This is a continuation of glibc bug 30037, whose root cause was not fixed: Missing cache information on x86-64 under Intel TDX glibc 2.34 and newer segfault if CPUID leaf 0x2 reports zero=20 Not sure if there is a public mailing list yet where TDX enablement is discussed. I'll point a few folks to this thread privately. The core of the issue is that CPUID.02H comes back as all zero. Current glibc uses CPUID.02H as the starting point to determine cache topology, so we get back all zeros: # getconf -a | grep CACHE LEVEL1_ICACHE_SIZE 0 LEVEL1_ICACHE_ASSOC=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 LEVEL1_ICACHE_LINESIZE 0 LEVEL1_DCACHE_SIZE 0 LEVEL1_DCACHE_ASSOC 0 LEVEL1_DCACHE_LINESIZE 0 LEVEL2_CACHE_SIZE 0 LEVEL2_CACHE_ASSOC 0 LEVEL2_CACHE_LINESIZE 0 LEVEL3_CACHE_SIZE 0 LEVEL3_CACHE_ASSOC 0 LEVEL3_CACHE_LINESIZE 0 LEVEL4_CACHE_SIZE 0 LEVEL4_CACHE_ASSOC=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 LEVEL4_CACHE_LINESIZE=20=20=20=20=20=20=20=20=20=20=20=20=20=20 This will almost certainly cause application hangs and crashes if they use the cache line sizes to divide up arrays for processing. Size 0 means that either no progress is made, or a division-by-zero trap occurs. (Full =E2=80=9Ccpuid -1 -r=E2=80=9D output below, from an Azure TDX instanc= e, shared with permission.) The current all-00H behavior is arguably not within the description of the Intel SDM because it lists special FEH and FFH descriptors to redirect to other CPUID information sources. (Current glibc only handles 0FFH redirects, apparently.) Some applications can get the cache information using those other means (ignoring CPUID.02H or using it as fallback only). Looking at Debian Code Search results, direct CPUID.02H are somewhat common: It looks like a few code snippets were copied around quite a bit. (You'd need to look at the context, though, to see if these applications are actually impacted.) I would prefer if this could be fixed in Intel TDX because it's the only the way we avoid additional userspace porting of applications, or at least impact analysis. But if TDX can't be fixed for some reason, we need to treat the all-00H as an instruction to glibc to gather the cache information by some other means. Thanks, Florian # /usr/bin/cpuid -1 -r=20=20 CPU: 0x00000000 0x00: eax=3D0x00000021 ebx=3D0x756e6547 ecx=3D0x6c65746e edx= =3D0x49656e69 0x00000001 0x00: eax=3D0x000806f8 ebx=3D0x00020800 ecx=3D0xfffaba17 edx= =3D0x1fabfbff 0x00000002 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000003 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000004 0x00: eax=3D0x00004121 ebx=3D0x02c0003f ecx=3D0x0000003f edx= =3D0x00000000 0x00000004 0x01: eax=3D0x00004122 ebx=3D0x01c0003f ecx=3D0x0000003f edx= =3D0x00000000 0x00000004 0x02: eax=3D0x00004143 ebx=3D0x03c0003f ecx=3D0x000007ff edx= =3D0x00000000 0x00000004 0x03: eax=3D0x00004163 ebx=3D0x0380003f ecx=3D0x0001bfff edx= =3D0x00000000 0x00000004 0x04: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000005 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000006 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000007 0x00: eax=3D0x00000001 ebx=3D0xf1bf2ff9 ecx=3D0x1b415fe6 edx= =3D0xffd14410 0x00000007 0x01: eax=3D0x00001c30 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000008 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000009 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000a 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000b 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000c 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000d 0x00: eax=3D0x000600e7 ebx=3D0x00002b00 ecx=3D0x00002b00 edx= =3D0x00000000 0x0000000d 0x01: eax=3D0x0000001f ebx=3D0x000029c0 ecx=3D0x00001800 edx= =3D0x00000000 0x0000000d 0x02: eax=3D0x00000100 ebx=3D0x00000240 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000d 0x05: eax=3D0x00000040 ebx=3D0x00000440 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000d 0x06: eax=3D0x00000200 ebx=3D0x00000480 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000d 0x07: eax=3D0x00000400 ebx=3D0x00000680 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000d 0x0b: eax=3D0x00000010 ebx=3D0x00000000 ecx=3D0x00000001 edx= =3D0x00000000 0x0000000d 0x0c: eax=3D0x00000018 ebx=3D0x00000000 ecx=3D0x00000001 edx= =3D0x00000000 0x0000000d 0x11: eax=3D0x00000040 ebx=3D0x00000ac0 ecx=3D0x00000002 edx= =3D0x00000000 0x0000000d 0x12: eax=3D0x00002000 ebx=3D0x00000b00 ecx=3D0x00000006 edx= =3D0x00000000 0x0000000e 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000000f 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000010 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000011 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000012 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000012 0x01: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000012 0x02: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000013 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000014 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000015 0x00: eax=3D0x00000001 ebx=3D0x00000054 ecx=3D0x017d7840 edx= =3D0x00000000 0x00000016 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000017 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000018 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000019 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001a 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001b 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001b 0x01: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001c 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001d 0x00: eax=3D0x00000001 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001d 0x01: eax=3D0x04002000 ebx=3D0x00080040 ecx=3D0x00000010 edx= =3D0x00000000 0x0000001e 0x00: eax=3D0x00000000 ebx=3D0x00004010 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001f 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x0000001f 0x01: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000020 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x00000021 0x00: eax=3D0x00000000 ebx=3D0x65746e49 ecx=3D0x20202020 edx= =3D0x5844546c 0x20000000 0x00: eax=3D0x00000000 ebx=3D0x65746e49 ecx=3D0x20202020 edx= =3D0x5844546c 0x40000000 0x00: eax=3D0x4000000c ebx=3D0x7263694d ecx=3D0x666f736f edx= =3D0x76482074 0x40000001 0x00: eax=3D0x31237648 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x40000002 0x00: eax=3D0x0000585d ebx=3D0x000a0000 ecx=3D0x00000001 edx= =3D0x00000447 0x40000003 0x00: eax=3D0x0000ae7f ebx=3D0x00628030 ecx=3D0x00000002 edx= =3D0xe0be47a2 0x40000004 0x00: eax=3D0x00360e24 ebx=3D0x00000fff ecx=3D0x00000034 edx= =3D0x00000000 0x40000005 0x00: eax=3D0x00000800 ebx=3D0x00000800 ecx=3D0x00009720 edx= =3D0x00000000 0x40000006 0x00: eax=3D0x0000000f ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x40000007 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x40000008 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x40000009 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x4000000a 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x4000000b 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x4000000c 0x00: eax=3D0x00000000 ebx=3D0x00000003 ecx=3D0x00000000 edx= =3D0x00000000 0x40000100 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000000 0x00: eax=3D0x80000008 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000001 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000121 edx= =3D0x2c100800 0x80000002 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000003 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000004 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000005 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000006 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000007 0x00: eax=3D0x00000000 ebx=3D0x00000000 ecx=3D0x00000000 edx= =3D0x00000000 0x80000008 0x00: eax=3D0x00003934 ebx=3D0x00000200 ecx=3D0x00000000 edx= =3D0x00000000 0x80860000 0x00: eax=3D0x00000000 ebx=3D0x65746e49 ecx=3D0x20202020 edx= =3D0x5844546c 0xc0000000 0x00: eax=3D0x00000000 ebx=3D0x65746e49 ecx=3D0x20202020 edx= =3D0x5844546c