From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oo1-xc2a.google.com (mail-oo1-xc2a.google.com [IPv6:2607:f8b0:4864:20::c2a]) by sourceware.org (Postfix) with ESMTPS id 6CF9B3858D38 for ; Mon, 11 Sep 2023 16:29:12 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 6CF9B3858D38 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-oo1-xc2a.google.com with SMTP id 006d021491bc7-573429f5874so2737945eaf.0 for ; Mon, 11 Sep 2023 09:29:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694449751; x=1695054551; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=z7VCiAruNykWCvhRWqp7Wwnun44edTthBAS2EUxsYo8=; b=Ddffu0aI4uiIdLnX0PKE0s3TpvaIO9I2JpNfher8rWNu3ghd1suQkNHz3L6/+TzIVd csUsn37qtWSlmFoMCoAvsPeHxDMF6UOmHhp7QIVZ/6V5a2tHmYXMnQkwILWipvNPkNmV PfCwXig41Hd8pGaGPwvc1LTGTyNVXtHHdI52yaq/e2T8Y1dhc//LkYOv3i7w5kJgB1g4 8oqtykc+YDodwzXPBkZd/ajVM7x9i1nEiGwGTSILb7MuM9r3HWEKluik5H5pQXCZUlVS GhGJ3wX3Y7JwwJ5IRPJoWmo4ib6o7YGSHP18FAE6cCJwnE6NoJ+8yjhGmEYeOR19sEDu LUug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694449751; x=1695054551; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z7VCiAruNykWCvhRWqp7Wwnun44edTthBAS2EUxsYo8=; b=jsffLL2JY8oP+rov0KWVRrYc1g1ihI5PY6hrQvNzzNa3p1CSAenAZ0qGPMu6rmhWu6 hHoN2LNR8uf14lRg3nvMKEuyCC+7JA6syW3ivGVX6VJkvkugqk/XilUtFOtSwrj154hp JRK7i/kPC1kqRZwYYZMGd3MRHB6q1tMXcmHpM+QXWmA/5NOaap6wghu8W23aHLDwn5rB UyC3FZYizHhiNYkks3ENSBEDllRAYaP2+upXNo8C7Iu9Ny4K4RGcWC9c+KWHiPrlaXLL 28CptJiucWV5mKH5obyZ1hC/Zbqz5cLprSfunYH5UdKk0XSYGdCV5d6N8maEu4go4QZw JK6w== X-Gm-Message-State: AOJu0YxUh7rKpaw8+wzBWkfaCf4tAwgpBnyPOFP+spwKewh5/FE3JWqr l7CIpugdqMezUrSANuuBA5kwUyV6ZSIxBRPQq7H0GGo/kjs= X-Google-Smtp-Source: AGHT+IGhsA1r44sLusV146YvEj/RT/6ZjAMzWtgNpXRIbIBsySdo/a76bI61DYt6OMphRaS0mAWxSwb4TjKuIJgfHNk= X-Received: by 2002:a05:6870:7029:b0:1bf:8df:2406 with SMTP id u41-20020a056870702900b001bf08df2406mr12800939oae.27.1694449751599; Mon, 11 Sep 2023 09:29:11 -0700 (PDT) MIME-Version: 1.0 References: <4a77d6294e0023338a8115fad9a3d549c47cae87.1694203757.git.fweimer@redhat.com> <871qf549sa.fsf@oldenburg.str.redhat.com> <87il8gptgo.fsf@oldenburg.str.redhat.com> In-Reply-To: <87il8gptgo.fsf@oldenburg.str.redhat.com> From: Noah Goldstein Date: Mon, 11 Sep 2023 11:28:54 -0500 Message-ID: Subject: Re: [PATCH 2/2] x86: Add generic CPUID data dumper to ld.so --list-diagnostics To: Florian Weimer Cc: libc-alpha@sourceware.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-3.3 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, Sep 11, 2023 at 11:26=E2=80=AFAM Florian Weimer wrote: > > * Noah Goldstein: > > > On Sun, Sep 10, 2023 at 11:24=E2=80=AFPM Florian Weimer wrote: > >> > >> * Noah Goldstein: > >> > >> > On Fri, Sep 8, 2023 at 3:10=E2=80=AFPM Florian Weimer wrote: > >> >> > >> >> This is surprisingly difficult to implement if the goal is to produ= ce > >> >> reasonably sized output. With the current approaches to output > >> >> compression (suppressing zeros and repeated results between CPUs, > >> >> folding ranges of identical subleaves, dealing with the %ecx > >> >> reflection issue), the output is less than 600 KiB even for systems > >> >> with 256 threads. > >> >> > >> > Maybe should just output a complete json? > >> > >> JSON cannot directly represent 64-bit integers, so it would need some > >> custom transformation for other parts of the --list-diagnostics output= . > >> > >> > Then users can pretty easily write scripts to extract the exact info= rmation > >> > they are after. Or or the dumper can be extended in the future to le= t > >> > the user specify fields/values to dump so it can be configured to be= more > >> > reasonable? > >> > >> I'm not sure what is unreasonable about the current implementation? I > >> complained about how hard it is getting the data and distilling it int= o > >> something that is not a gigantic data blob. > >> > >> To be clear, without only trivial zero-values suppression, brute-force > >> enumeration (cutting off at 512 subleaves) results in roughly 8 KiB of > >> raw data per *CPU*. It's even larger for recent CPUs which have more = of > >> the funny ECX behavior (where unsupported subleaves do not come back a= s > >> zero). > > > > Maybe I misunderstand but the commit message is saying a 256 core syste= m > > dumps 600KB? > > For all CPUs, after hex encoding. So that's more like 1,000 bytes of > data per CPU (and the 8 KiB number was just an estimate, for two ECX > runaways, if you have more of those the number grows quickly). > > > If so, that seems like a lot for a person to just grok hence why I'd > > favor a standardized format. > > Sure, this calls out for automated processing. We have Python parsing > code in the testsuite, which could be repurposed. > > > If JSON isn't really feasible for technical reasons, however, so be it. > > There is that 53 bit problem, and we'd still have to use string keys for > the objects. We can't find meaningful names for the information? If not do we even want to dump it? Although I'm probably trivializing the problem, a quick glance at: https://github.com/google/cpu_features and there json dump skips everything but feature flags. > > Thanks, > Florian >