Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump note

[John Baldwin <jhb@FreeBSD.org>]

On 10/26/23 9:18 AM, George, Jini Susan wrote:
> [AMD Official Use Only - General]
> 
> I think it might be better to go with different NT_ values for the P and the E cores. We will also need to put in additional mapping information as to which core/ cpu id belongs to which core set (P or E).

Hmm, do you mean mapping thread (LWP) IDs to specific core sets?
Right now core dumps don't really include the mapping of threads
to individual CPU cores, in part because threads can migrate
between cores (subject to administrative and application constraints).
I'm not really sure there is a use case for storing that information.

One way you could store that would be to have a per-thread note
storing the logical CPU mask (or cpuset, not sure which term Linux
uses, FreeBSD uses cpuset) for each thread, but then you'd need
perhaps some other note mapping logical CPU IDs to other CPU data.
(For example, if you did have a separate NT_X86_CPUID_E_CORE, you
could have some other note that was a short table mapping CPU IDs
to CPUID notes (e.g. cpu 0 is NT_X86_CPUID, cpu 1 is NT_X86_CPUID,
cpu 2 is NT_X86_CPUID_E_CORE).

However, I'm not sure it makes sense to store all of that now without
a use case.  I do think though that this does show there are ways to
extend the information in the future should a use case arrive.

> Thanks
> Jini.
> 
>>> -----Original Message-----
>>> From: John Baldwin <jhb@FreeBSD.org>
>>> Sent: Tuesday, October 17, 2023 6:06 AM
>>> To: George, Jini Susan <JiniSusan.George@amd.com>; Simon Marchi
>>> <simon.marchi@polymtl.ca>; gdb-patches@sourceware.org
>>> Cc: Felix <felix.willgerodt@intel.com>; Balasubrmanian, Vignesh
>>> <Vignesh.Balasubrmanian@amd.com>
>>> Subject: Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump note
>>>
>>> Caution: This message originated from an External Source. Use proper caution
>>> when opening attachments, clicking links, or responding.
>>>
>>>
>>> On 10/13/23 2:38 AM, George, Jini Susan wrote:
>>>> [AMD Official Use Only - General]
>>>>
>>>> I think that even if the xsave layout remains uniform across the cores of a
>>> system, since we are trying to design an extensible .note section which can
>>> possibly hold all kinds of CPUID information, we might want to consider various
>>> scenarios wherein the CPUID information might differ across cores (esp for
>>> big.LITTLE/(P/E)), like the cache information, perhaps ? It might be more prudent
>>> to include the coretype information also in such cases ?
>>>
>>> It's certainly occurred to me that it might be prudent to include the full
>>> complement of CPUID leaves even in the initial version of this note should that
>>> information prove useful in the future.
>>>
>>> However, I'm not quite sure how we should "name" the different CPUID leaf sets
>>> in this case (e.g. the set for E cores vs the set for P cores).  In particular, it's not
>>> quite clear to me how many sets future systems might have?  One route is to use
>>> additional NT_ values for different sets, (e.g. a NT_X86_CPUID_E_CORE or some
>>> such if the "default" set is for the P core).  We could also embed an identifier at
>>> the start of a note, e.g.
>>> a 32-bit integer, and dump one note per set, so you have always have a note for
>>> set 0, but in a system with E cores you might have a set 1 where the convention
>>> would be P cores are set 0, and E cores are set 1.  Future systems with multiple
>>> core types would have to decide what type of mapping to use between core
>>> types and set IDs.  One issue with the second approach is if you want these
>>> notes accessible via ptrace() and not just in cores, it wouldn't be clear how to
>>> request the leaves for set N if they all share the NT_* value.
>>>
>>>> Rgds
>>>> Jini.
>>>>
>>>>>> -----Original Message-----
>>>>>> From: John Baldwin <jhb@FreeBSD.org>
>>>>>> Sent: Thursday, October 12, 2023 10:48 PM
>>>>>> To: Simon Marchi <simon.marchi@polymtl.ca>;
>>>>>> gdb-patches@sourceware.org
>>>>>> Cc: Felix <felix.willgerodt@intel.com>; George, Jini Susan
>>>>>> <JiniSusan.George@amd.com>
>>>>>> Subject: Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump
>>>>>> note
>>>>>>
>>>>>> Caution: This message originated from an External Source. Use proper
>>>>>> caution when opening attachments, clicking links, or responding.
>>>>>>
>>>>>>
>>>>>> On 10/12/23 7:33 AM, Simon Marchi wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 2023-10-09 14:36, John Baldwin wrote:
>>>>>>>> One of the shortcomings of the previous XSAVE patch series is that
>>>>>>>> it depends on heuristics based on the total XSAVE register set
>>>>>>>> size and
>>>>>>>> XCR0 mask to infer layouts of the various register blocks for core
>>>>>>>> dumps.  This series introduces a new x86-specific core dump note
>>>>>>>> intended to supplant these heuristics by storing the raw CPUID
>>>>>>>> leaves describing the XSAVE layout in core dumps.
>>>>>>>>
>>>>>>>> This series proposes a new core dump note, NT_X86_CPUID (0x205),
>>>>>>>> which contains an array of structures.  Each structure describes
>>>>>>>> an invidual CPUID sub-leaf containing both the inputs to CPUID
>>>>>>>> (%eax and
>>>>>>>> %ecx) and the outputs (%eax, %ebx, %ecx, and %edx) in a format
>>>>>>>> roughly matching the follow C structure:
>>>>>>>>
>>>>>>>> struct cpuid_leaf
>>>>>>>> {
>>>>>>>>        uint32_t leaf;
>>>>>>>>        uint32_t subleaf;
>>>>>>>>        uint32_t eax;
>>>>>>>>        uint32_t ebx;
>>>>>>>>        uint32_t ecx;
>>>>>>>>        uint32_t edx;
>>>>>>>> };
>>>>>>>>
>>>>>>>> This format is not XSAVE-specific and implementations could choose
>>>>>>>> to add additional CPUID leaves to this structure if needed in the future.
>>>>>>>> Consumers of this note should lookup the value of required leaves
>>>>>>>> and ignore any unneeded leaves.
>>>>>>>>
>>>>>>>> An alternate approach might be to write out a more XSAVE-specific
>>>>>>>> note that is an array containing the offset and size of each XSAVE region.
>>>>>>>
>>>>>>> Something I thought about: I think there are asymmetrical x86
>>>>>>> processors now, with "big" and "little" cores, not sure how they
>>>>>>> call them.  For them, is it possible for the XSAVE layout to be
>>>>>>> different per core, for instance some cores supporting AVX512 and
>>>>>>> some cores not?  And therefore, will we eventually need to include
>>>>>>> CPUID / XSAVE information for more than one CPU core type in the core
>>> file notes?
>>>>>>
>>>>>> (The Intel names are "P" (performance) and "E" (energy-efficient)
>>>>>> cores
>>>>>> btw)
>>>>>>
>>>>>> I have no idea if they use the same or different XSAVE layouts.  It
>>>>>> would seem to be a royal pain if they did as everytime a user thread
>>>>>> migrated between cores the OS would have to convert the XSAVE block
>> >from one layout to the other.
>>>>>> FreeBSD does not have any notion of multiple layouts today and just
>>>>>> assumes that the XSAVE layout is uniform across all cores in a
>>>>>> system.  I believe Linux's kernel does the same from my reading.
>>>>>>
>>>>>> --
>>>>>> John Baldwin
>>>>
>>>
>>> --
>>> John Baldwin
> 

-- 
John Baldwin