Re: [PATCH 2/2] x86: Add generic CPUID data dumper to ld.so --list-diagnostics

[Noah Goldstein <goldstein.w.n@gmail.com>]

On Mon, Sep 11, 2023 at 11:26 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Noah Goldstein:
>
> > On Sun, Sep 10, 2023 at 11:24 PM Florian Weimer <fweimer@redhat.com> wrote:
> >>
> >> * Noah Goldstein:
> >>
> >> > On Fri, Sep 8, 2023 at 3:10 PM Florian Weimer <fweimer@redhat.com> wrote:
> >> >>
> >> >> This is surprisingly difficult to implement if the goal is to produce
> >> >> reasonably sized output.  With the current approaches to output
> >> >> compression (suppressing zeros and repeated results between CPUs,
> >> >> folding ranges of identical subleaves, dealing with the %ecx
> >> >> reflection issue), the output is less than 600 KiB even for systems
> >> >> with 256 threads.
> >> >>
> >> > Maybe should just output a complete json?
> >>
> >> JSON cannot directly represent 64-bit integers, so it would need some
> >> custom transformation for other parts of the --list-diagnostics output.
> >>
> >> > Then users can pretty easily write scripts to extract the exact information
> >> > they are after. Or or the dumper can be extended in the future to let
> >> > the user specify fields/values to dump so it can be configured to be more
> >> > reasonable?
> >>
> >> I'm not sure what is unreasonable about the current implementation?  I
> >> complained about how hard it is getting the data and distilling it into
> >> something that is not a gigantic data blob.
> >>
> >> To be clear, without only trivial zero-values suppression, brute-force
> >> enumeration (cutting off at 512 subleaves) results in roughly 8 KiB of
> >> raw data per *CPU*.  It's even larger for recent CPUs which have more of
> >> the funny ECX behavior (where unsupported subleaves do not come back as
> >> zero).
> >
> > Maybe I misunderstand but the commit message is saying a 256 core system
> > dumps 600KB?
>
> For all CPUs, after hex encoding.  So that's more like 1,000 bytes of
> data per CPU (and the 8 KiB number was just an estimate, for two ECX
> runaways, if you have more of those the number grows quickly).
>
> > If so, that seems like a lot for a person to just grok hence why I'd
> > favor a standardized format.
>
> Sure, this calls out for automated processing.  We have Python parsing
> code in the testsuite, which could be repurposed.
>
> > If JSON isn't really feasible for technical reasons, however, so be it.
>
> There is that 53 bit problem, and we'd still have to use string keys for
> the objects.

We can't find meaningful names for the information? If not do we even
want to dump it?

Although I'm probably trivializing the problem, a quick glance at:
https://github.com/google/cpu_features
and there json dump skips everything but feature flags.
>
> Thanks,
> Florian
>