From: Florian Weimer <fweimer@redhat.com>
To: libc-alpha@sourceware.org, qemu-devel@nongnu.org
Cc: "H. Peter Anvin" <hpa@zytor.com>, Vitaly Kuznetsov <vkuznets@redhat.com>
Subject: Missing cache information on x86-64 under Intel TDX (glibc bug 30643)
Date: Mon, 17 Jul 2023 10:38:50 +0200 [thread overview]
Message-ID: <87mszv7x0l.fsf@oldenburg.str.redhat.com> (raw)
This is a continuation of glibc bug 30037, whose root cause was not
fixed:
Missing cache information on x86-64 under Intel TDX
<https://sourceware.org/bugzilla/show_bug.cgi?id=30643>
glibc 2.34 and newer segfault if CPUID leaf 0x2 reports zero
<https://sourceware.org/bugzilla/show_bug.cgi?id=30037>
Not sure if there is a public mailing list yet where TDX enablement is
discussed. I'll point a few folks to this thread privately.
The core of the issue is that CPUID.02H comes back as all zero. Current
glibc uses CPUID.02H as the starting point to determine cache topology,
so we get back all zeros:
# getconf -a | grep CACHE
LEVEL1_ICACHE_SIZE 0
LEVEL1_ICACHE_ASSOC
LEVEL1_ICACHE_LINESIZE 0
LEVEL1_DCACHE_SIZE 0
LEVEL1_DCACHE_ASSOC 0
LEVEL1_DCACHE_LINESIZE 0
LEVEL2_CACHE_SIZE 0
LEVEL2_CACHE_ASSOC 0
LEVEL2_CACHE_LINESIZE 0
LEVEL3_CACHE_SIZE 0
LEVEL3_CACHE_ASSOC 0
LEVEL3_CACHE_LINESIZE 0
LEVEL4_CACHE_SIZE 0
LEVEL4_CACHE_ASSOC
LEVEL4_CACHE_LINESIZE
This will almost certainly cause application hangs and crashes if they
use the cache line sizes to divide up arrays for processing. Size 0
means that either no progress is made, or a division-by-zero trap
occurs.
(Full “cpuid -1 -r” output below, from an Azure TDX instance, shared
with permission.)
The current all-00H behavior is arguably not within the description of
the Intel SDM because it lists special FEH and FFH descriptors to
redirect to other CPUID information sources. (Current glibc only
handles 0FFH redirects, apparently.) Some applications can get the
cache information using those other means (ignoring CPUID.02H or using
it as fallback only). Looking at Debian Code Search results, direct
CPUID.02H are somewhat common:
<https://codesearch.debian.net/search?q=cpuid%5Cs*%5C%282%2C&literal=0&perpkg=1>
It looks like a few code snippets were copied around quite a bit.
(You'd need to look at the context, though, to see if these applications
are actually impacted.)
I would prefer if this could be fixed in Intel TDX because it's the only
the way we avoid additional userspace porting of applications, or at
least impact analysis. But if TDX can't be fixed for some reason, we
need to treat the all-00H as an instruction to glibc to gather the cache
information by some other means.
Thanks,
Florian
# /usr/bin/cpuid -1 -r
CPU:
0x00000000 0x00: eax=0x00000021 ebx=0x756e6547 ecx=0x6c65746e edx=0x49656e69
0x00000001 0x00: eax=0x000806f8 ebx=0x00020800 ecx=0xfffaba17 edx=0x1fabfbff
0x00000002 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000003 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000004 0x00: eax=0x00004121 ebx=0x02c0003f ecx=0x0000003f edx=0x00000000
0x00000004 0x01: eax=0x00004122 ebx=0x01c0003f ecx=0x0000003f edx=0x00000000
0x00000004 0x02: eax=0x00004143 ebx=0x03c0003f ecx=0x000007ff edx=0x00000000
0x00000004 0x03: eax=0x00004163 ebx=0x0380003f ecx=0x0001bfff edx=0x00000000
0x00000004 0x04: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000005 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000006 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000007 0x00: eax=0x00000001 ebx=0xf1bf2ff9 ecx=0x1b415fe6 edx=0xffd14410
0x00000007 0x01: eax=0x00001c30 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000008 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000009 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000000a 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000000b 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000000c 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000000d 0x00: eax=0x000600e7 ebx=0x00002b00 ecx=0x00002b00 edx=0x00000000
0x0000000d 0x01: eax=0x0000001f ebx=0x000029c0 ecx=0x00001800 edx=0x00000000
0x0000000d 0x02: eax=0x00000100 ebx=0x00000240 ecx=0x00000000 edx=0x00000000
0x0000000d 0x05: eax=0x00000040 ebx=0x00000440 ecx=0x00000000 edx=0x00000000
0x0000000d 0x06: eax=0x00000200 ebx=0x00000480 ecx=0x00000000 edx=0x00000000
0x0000000d 0x07: eax=0x00000400 ebx=0x00000680 ecx=0x00000000 edx=0x00000000
0x0000000d 0x0b: eax=0x00000010 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
0x0000000d 0x0c: eax=0x00000018 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
0x0000000d 0x11: eax=0x00000040 ebx=0x00000ac0 ecx=0x00000002 edx=0x00000000
0x0000000d 0x12: eax=0x00002000 ebx=0x00000b00 ecx=0x00000006 edx=0x00000000
0x0000000e 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000000f 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000010 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000011 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000012 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000012 0x01: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000012 0x02: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000013 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000014 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000015 0x00: eax=0x00000001 ebx=0x00000054 ecx=0x017d7840 edx=0x00000000
0x00000016 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000017 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000018 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000019 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001a 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001b 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001b 0x01: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001c 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001d 0x00: eax=0x00000001 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001d 0x01: eax=0x04002000 ebx=0x00080040 ecx=0x00000010 edx=0x00000000
0x0000001e 0x00: eax=0x00000000 ebx=0x00004010 ecx=0x00000000 edx=0x00000000
0x0000001f 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x0000001f 0x01: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000020 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x00000021 0x00: eax=0x00000000 ebx=0x65746e49 ecx=0x20202020 edx=0x5844546c
0x20000000 0x00: eax=0x00000000 ebx=0x65746e49 ecx=0x20202020 edx=0x5844546c
0x40000000 0x00: eax=0x4000000c ebx=0x7263694d ecx=0x666f736f edx=0x76482074
0x40000001 0x00: eax=0x31237648 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x40000002 0x00: eax=0x0000585d ebx=0x000a0000 ecx=0x00000001 edx=0x00000447
0x40000003 0x00: eax=0x0000ae7f ebx=0x00628030 ecx=0x00000002 edx=0xe0be47a2
0x40000004 0x00: eax=0x00360e24 ebx=0x00000fff ecx=0x00000034 edx=0x00000000
0x40000005 0x00: eax=0x00000800 ebx=0x00000800 ecx=0x00009720 edx=0x00000000
0x40000006 0x00: eax=0x0000000f ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x40000007 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x40000008 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x40000009 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x4000000a 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x4000000b 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x4000000c 0x00: eax=0x00000000 ebx=0x00000003 ecx=0x00000000 edx=0x00000000
0x40000100 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000000 0x00: eax=0x80000008 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000001 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000121 edx=0x2c100800
0x80000002 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000003 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000004 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000005 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000006 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000007 0x00: eax=0x00000000 ebx=0x00000000 ecx=0x00000000 edx=0x00000000
0x80000008 0x00: eax=0x00003934 ebx=0x00000200 ecx=0x00000000 edx=0x00000000
0x80860000 0x00: eax=0x00000000 ebx=0x65746e49 ecx=0x20202020 edx=0x5844546c
0xc0000000 0x00: eax=0x00000000 ebx=0x65746e49 ecx=0x20202020 edx=0x5844546c
reply other threads:[~2023-07-17 8:38 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87mszv7x0l.fsf@oldenburg.str.redhat.com \
--to=fweimer@redhat.com \
--cc=hpa@zytor.com \
--cc=libc-alpha@sourceware.org \
--cc=qemu-devel@nongnu.org \
--cc=vkuznets@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).