From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oo1-xc32.google.com (mail-oo1-xc32.google.com [IPv6:2607:f8b0:4864:20::c32]) by sourceware.org (Postfix) with ESMTPS id 56FCA385841D for ; Mon, 11 Sep 2023 16:31:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 56FCA385841D Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-oo1-xc32.google.com with SMTP id 006d021491bc7-5738949f62cso3016069eaf.0 for ; Mon, 11 Sep 2023 09:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1694449888; x=1695054688; darn=sourceware.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=idLCEzAF6Bg0clgDkDPTfmfa+OCEUWG/KG6W4T51aaA=; b=JQdDRRSQ86MGaVnnmvJg/WBW6kmxmJDFllogLd99l2oO7kk/OzYe3+xOEeg4SdEOEc YUfHmhysanJ7zjyYc+2VuAjwwvnCl6ACpnXKb/K8IbG1oXJigb8MdQAm4/rbUsvpPUum Aq1QnDWYnxLWeJiQ5tipOs1zLduG7xun2poM01ThwuoA3qLrcjrHhjv2q0YLJJU/k8Hx bP8ugpKUnU6WYpoMFGySuQJ4C89kZzw908VcL4Cyg+Fvtg2xyoxm3PtyRbx8+BA3eRdZ OeZfm7DacJdhcnVjI1yfVVBe4tXNgb/yRndAUBa95uFZGJlcF2Ck7oYIZ/tb5jmHf2mt 09LQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694449888; x=1695054688; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=idLCEzAF6Bg0clgDkDPTfmfa+OCEUWG/KG6W4T51aaA=; b=dn8Y/XJ4x3Kpo/ONmkfAElBuTY3eA03NIeymFU1cK38VRi42K8WTcTwdc73fFbXKyp DW01X8N2ZR5r8rbKnqG+64QUWN1eolx5+KnEGHfoBpaIEpA4RtZrQv0ydnuaZGoKUdYA BCs1RoGExe3BCTfHV8sQGVjijbp6v1agCVUD4TTFDSuSgHGyTsIAa/gsIDEQ0F/TKCUX mX+Qp6sP/lhz4pL8ZBr6Ymb+9qPvsKideVZjv1wbLxRpOO6Mz6R7MW2F7Deq5TgjoxZX YTIvDEi/q4Q35KP6MR1/x5HeyRgy3SE5BbCBHfzGn1Wh/f3mzUxHjW4FsWfp++gWTQl0 lVpA== X-Gm-Message-State: AOJu0Yyb7TLW5rI83jbzXeHZrdXZekMMPgej4tfYiY5UXZHg2/eXuQDO Nbp39lRdrMbCsbtTi14STKFvggb1ii6KMHQ/4th1LlZM X-Google-Smtp-Source: AGHT+IEdWByHvJP/x4yAUJc8hAMkXMTLcyunOPwmFdzMda1UYRIbNMCSav3Y6hE+vpeoDlzavB2ke4VJV14ypWF6HSE= X-Received: by 2002:a05:6870:d202:b0:1b0:3d61:553e with SMTP id g2-20020a056870d20200b001b03d61553emr12292927oac.15.1694449888574; Mon, 11 Sep 2023 09:31:28 -0700 (PDT) MIME-Version: 1.0 References: <4a77d6294e0023338a8115fad9a3d549c47cae87.1694203757.git.fweimer@redhat.com> <871qf549sa.fsf@oldenburg.str.redhat.com> <87il8gptgo.fsf@oldenburg.str.redhat.com> In-Reply-To: From: Noah Goldstein Date: Mon, 11 Sep 2023 11:31:12 -0500 Message-ID: Subject: Re: [PATCH 2/2] x86: Add generic CPUID data dumper to ld.so --list-diagnostics To: Florian Weimer Cc: libc-alpha@sourceware.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, Sep 11, 2023 at 11:28=E2=80=AFAM Noah Goldstein wrote: > > On Mon, Sep 11, 2023 at 11:26=E2=80=AFAM Florian Weimer wrote: > > > > * Noah Goldstein: > > > > > On Sun, Sep 10, 2023 at 11:24=E2=80=AFPM Florian Weimer wrote: > > >> > > >> * Noah Goldstein: > > >> > > >> > On Fri, Sep 8, 2023 at 3:10=E2=80=AFPM Florian Weimer wrote: > > >> >> > > >> >> This is surprisingly difficult to implement if the goal is to pro= duce > > >> >> reasonably sized output. With the current approaches to output > > >> >> compression (suppressing zeros and repeated results between CPUs, > > >> >> folding ranges of identical subleaves, dealing with the %ecx > > >> >> reflection issue), the output is less than 600 KiB even for syste= ms > > >> >> with 256 threads. > > >> >> > > >> > Maybe should just output a complete json? > > >> > > >> JSON cannot directly represent 64-bit integers, so it would need som= e > > >> custom transformation for other parts of the --list-diagnostics outp= ut. > > >> > > >> > Then users can pretty easily write scripts to extract the exact in= formation > > >> > they are after. Or or the dumper can be extended in the future to = let > > >> > the user specify fields/values to dump so it can be configured to = be more > > >> > reasonable? > > >> > > >> I'm not sure what is unreasonable about the current implementation? = I > > >> complained about how hard it is getting the data and distilling it i= nto > > >> something that is not a gigantic data blob. > > >> > > >> To be clear, without only trivial zero-values suppression, brute-for= ce > > >> enumeration (cutting off at 512 subleaves) results in roughly 8 KiB = of > > >> raw data per *CPU*. It's even larger for recent CPUs which have mor= e of > > >> the funny ECX behavior (where unsupported subleaves do not come back= as > > >> zero). > > > > > > Maybe I misunderstand but the commit message is saying a 256 core sys= tem > > > dumps 600KB? > > > > For all CPUs, after hex encoding. So that's more like 1,000 bytes of > > data per CPU (and the 8 KiB number was just an estimate, for two ECX > > runaways, if you have more of those the number grows quickly). > > > > > If so, that seems like a lot for a person to just grok hence why I'd > > > favor a standardized format. > > > > Sure, this calls out for automated processing. We have Python parsing > > code in the testsuite, which could be repurposed. > > > > > If JSON isn't really feasible for technical reasons, however, so be i= t. > > > > There is that 53 bit problem, and we'd still have to use string keys fo= r > > the objects. > > We can't find meaningful names for the information? If not do we even > want to dump it? > > Although I'm probably trivializing the problem, a quick glance at: > https://github.com/google/cpu_features > and there json dump skips everything but feature flags. Thats not right actually, they dump itlb/cache info but don't do per-thread= . > > > > Thanks, > > Florian > >