RE: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump note

["George, Jini Susan" <JiniSusan.George@amd.com>]

[Public]

If tomorrow, we have to deal with varying xsave layouts/ cache or any other information for different coresets, then we would need the mapping from LWPIDs to coresets.

>One way you could store that would be to have a per-thread note storing the
>logical CPU mask (or cpuset, not sure which term Linux uses, FreeBSD uses
>cpuset) for each thread, but then you'd need perhaps some other note mapping
>logical CPU IDs to other CPU data.
>(For example, if you did have a separate NT_X86_CPUID_E_CORE, you could
>have some other note that was a short table mapping CPU IDs to CPUID notes
>(e.g. cpu 0 is NT_X86_CPUID, cpu 1 is NT_X86_CPUID, cpu 2 is
>NT_X86_CPUID_E_CORE).

I was thinking of having the per-thread note storing the last executed cpu for the thread -- not sure if the cpuset will be needed, but I could be missing something. But yes, we will need 3 extra note types for P/E scenarios. (the per process E note, the per process cpu ID to coreset mapping note and the per thread last executed cpu/ cpuset note).

So, yes, since we can add other required information as need be as extra notes in the future, I feel none of this should hinder the current patch with going through.

Thanks,
Jini.

>>-----Original Message-----
>>From: John Baldwin <jhb@FreeBSD.org>
>>Sent: Friday, October 27, 2023 8:23 AM
>>To: George, Jini Susan <JiniSusan.George@amd.com>; Simon Marchi
>><simon.marchi@polymtl.ca>; gdb-patches@sourceware.org
>>Cc: Felix <felix.willgerodt@intel.com>; Balasubrmanian, Vignesh
>><Vignesh.Balasubrmanian@amd.com>
>>Subject: Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump note
>>
>>Caution: This message originated from an External Source. Use proper caution
>>when opening attachments, clicking links, or responding.
>>
>>
>>On 10/26/23 9:18 AM, George, Jini Susan wrote:
>>> [AMD Official Use Only - General]
>>>
>>> I think it might be better to go with different NT_ values for the P and the E
>>cores. We will also need to put in additional mapping information as to which
>>core/ cpu id belongs to which core set (P or E).
>>
>>Hmm, do you mean mapping thread (LWP) IDs to specific core sets?
>>Right now core dumps don't really include the mapping of threads to individual
>>CPU cores, in part because threads can migrate between cores (subject to
>>administrative and application constraints).
>>I'm not really sure there is a use case for storing that information.
>>
>>One way you could store that would be to have a per-thread note storing the
>>logical CPU mask (or cpuset, not sure which term Linux uses, FreeBSD uses
>>cpuset) for each thread, but then you'd need perhaps some other note mapping
>>logical CPU IDs to other CPU data.
>>(For example, if you did have a separate NT_X86_CPUID_E_CORE, you could
>>have some other note that was a short table mapping CPU IDs to CPUID notes
>>(e.g. cpu 0 is NT_X86_CPUID, cpu 1 is NT_X86_CPUID, cpu 2 is
>>NT_X86_CPUID_E_CORE).
>>
>>However, I'm not sure it makes sense to store all of that now without a use
>>case.  I do think though that this does show there are ways to extend the
>>information in the future should a use case arrive.
>>
>>> Thanks
>>> Jini.
>>>
>>>>> -----Original Message-----
>>>>> From: John Baldwin <jhb@FreeBSD.org>
>>>>> Sent: Tuesday, October 17, 2023 6:06 AM
>>>>> To: George, Jini Susan <JiniSusan.George@amd.com>; Simon Marchi
>>>>> <simon.marchi@polymtl.ca>; gdb-patches@sourceware.org
>>>>> Cc: Felix <felix.willgerodt@intel.com>; Balasubrmanian, Vignesh
>>>>> <Vignesh.Balasubrmanian@amd.com>
>>>>> Subject: Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core dump
>>>>> note
>>>>>
>>>>> Caution: This message originated from an External Source. Use proper
>>>>> caution when opening attachments, clicking links, or responding.
>>>>>
>>>>>
>>>>> On 10/13/23 2:38 AM, George, Jini Susan wrote:
>>>>>> [AMD Official Use Only - General]
>>>>>>
>>>>>> I think that even if the xsave layout remains uniform across the
>>>>>> cores of a
>>>>> system, since we are trying to design an extensible .note section
>>>>> which can possibly hold all kinds of CPUID information, we might
>>>>> want to consider various scenarios wherein the CPUID information
>>>>> might differ across cores (esp for big.LITTLE/(P/E)), like the cache
>>>>> information, perhaps ? It might be more prudent to include the coretype
>>information also in such cases ?
>>>>>
>>>>> It's certainly occurred to me that it might be prudent to include
>>>>> the full complement of CPUID leaves even in the initial version of
>>>>> this note should that information prove useful in the future.
>>>>>
>>>>> However, I'm not quite sure how we should "name" the different CPUID
>>>>> leaf sets in this case (e.g. the set for E cores vs the set for P
>>>>> cores).  In particular, it's not quite clear to me how many sets
>>>>> future systems might have?  One route is to use additional NT_
>>>>> values for different sets, (e.g. a NT_X86_CPUID_E_CORE or some such
>>>>> if the "default" set is for the P core).  We could also embed an identifier at
>>the start of a note, e.g.
>>>>> a 32-bit integer, and dump one note per set, so you have always have
>>>>> a note for set 0, but in a system with E cores you might have a set
>>>>> 1 where the convention would be P cores are set 0, and E cores are
>>>>> set 1.  Future systems with multiple core types would have to decide
>>>>> what type of mapping to use between core types and set IDs.  One
>>>>> issue with the second approach is if you want these notes accessible
>>>>> via ptrace() and not just in cores, it wouldn't be clear how to request the
>>leaves for set N if they all share the NT_* value.
>>>>>
>>>>>> Rgds
>>>>>> Jini.
>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: John Baldwin <jhb@FreeBSD.org>
>>>>>>>> Sent: Thursday, October 12, 2023 10:48 PM
>>>>>>>> To: Simon Marchi <simon.marchi@polymtl.ca>;
>>>>>>>> gdb-patches@sourceware.org
>>>>>>>> Cc: Felix <felix.willgerodt@intel.com>; George, Jini Susan
>>>>>>>> <JiniSusan.George@amd.com>
>>>>>>>> Subject: Re: [RFC 00/13] Proposal for a new NT_X86_CPUID core
>>>>>>>> dump note
>>>>>>>>
>>>>>>>> Caution: This message originated from an External Source. Use
>>>>>>>> proper caution when opening attachments, clicking links, or responding.
>>>>>>>>
>>>>>>>>
>>>>>>>> On 10/12/23 7:33 AM, Simon Marchi wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 2023-10-09 14:36, John Baldwin wrote:
>>>>>>>>>> One of the shortcomings of the previous XSAVE patch series is
>>>>>>>>>> that it depends on heuristics based on the total XSAVE register
>>>>>>>>>> set size and
>>>>>>>>>> XCR0 mask to infer layouts of the various register blocks for
>>>>>>>>>> core dumps.  This series introduces a new x86-specific core
>>>>>>>>>> dump note intended to supplant these heuristics by storing the
>>>>>>>>>> raw CPUID leaves describing the XSAVE layout in core dumps.
>>>>>>>>>>
>>>>>>>>>> This series proposes a new core dump note, NT_X86_CPUID
>>>>>>>>>> (0x205), which contains an array of structures.  Each structure
>>>>>>>>>> describes an invidual CPUID sub-leaf containing both the inputs
>>>>>>>>>> to CPUID (%eax and
>>>>>>>>>> %ecx) and the outputs (%eax, %ebx, %ecx, and %edx) in a format
>>>>>>>>>> roughly matching the follow C structure:
>>>>>>>>>>
>>>>>>>>>> struct cpuid_leaf
>>>>>>>>>> {
>>>>>>>>>>        uint32_t leaf;
>>>>>>>>>>        uint32_t subleaf;
>>>>>>>>>>        uint32_t eax;
>>>>>>>>>>        uint32_t ebx;
>>>>>>>>>>        uint32_t ecx;
>>>>>>>>>>        uint32_t edx;
>>>>>>>>>> };
>>>>>>>>>>
>>>>>>>>>> This format is not XSAVE-specific and implementations could
>>>>>>>>>> choose to add additional CPUID leaves to this structure if needed in
>>the future.
>>>>>>>>>> Consumers of this note should lookup the value of required
>>>>>>>>>> leaves and ignore any unneeded leaves.
>>>>>>>>>>
>>>>>>>>>> An alternate approach might be to write out a more
>>>>>>>>>> XSAVE-specific note that is an array containing the offset and size of
>>each XSAVE region.
>>>>>>>>>
>>>>>>>>> Something I thought about: I think there are asymmetrical x86
>>>>>>>>> processors now, with "big" and "little" cores, not sure how they
>>>>>>>>> call them.  For them, is it possible for the XSAVE layout to be
>>>>>>>>> different per core, for instance some cores supporting AVX512
>>>>>>>>> and some cores not?  And therefore, will we eventually need to
>>>>>>>>> include CPUID / XSAVE information for more than one CPU core
>>>>>>>>> type in the core
>>>>> file notes?
>>>>>>>>
>>>>>>>> (The Intel names are "P" (performance) and "E" (energy-efficient)
>>>>>>>> cores
>>>>>>>> btw)
>>>>>>>>
>>>>>>>> I have no idea if they use the same or different XSAVE layouts.
>>>>>>>> It would seem to be a royal pain if they did as everytime a user
>>>>>>>> thread migrated between cores the OS would have to convert the
>>>>>>>> XSAVE block
>>>> >from one layout to the other.
>>>>>>>> FreeBSD does not have any notion of multiple layouts today and
>>>>>>>> just assumes that the XSAVE layout is uniform across all cores in
>>>>>>>> a system.  I believe Linux's kernel does the same from my reading.
>>>>>>>>
>>>>>>>> --
>>>>>>>> John Baldwin
>>>>>>
>>>>>
>>>>> --
>>>>> John Baldwin
>>>
>>
>>--
>>John Baldwin