From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wr1-f53.google.com (mail-wr1-f53.google.com [209.85.221.53]) by sourceware.org (Postfix) with ESMTPS id 9DEC43959E57; Tue, 4 May 2021 13:43:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 9DEC43959E57 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=debian.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=luca.boccassi@gmail.com Received: by mail-wr1-f53.google.com with SMTP id n2so9524443wrm.0; Tue, 04 May 2021 06:43:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version; bh=Z5ZvvQ9ouX9nmXu7aYaeip6FZWfy23T04d3ogr2BO7I=; b=dLPyS/MnoUjgC6VsOIy6ZfAptZij3i3YoppwDN3uYzVCKUdL8AdDl0CrAeddMoq6n1 o/ctvISP8v1Ham3NNhLZGdrbLkIlGQxla7hiM5MMSLQobrQYjHtzyWZVVazPeC0qUJoC CQgU0wCaE+8idSibpyLBFcPt+6wP4Who3dV/1+eoJ0erT9CiyjfWd8xi6Df9vE0Refqx OmvIGK0FdmWLY8+pSgWzCyB/6BaVxNL90vn80YQ9CO8zffzbTXb/MY5G7rB4MDxFkKrs evb22SzH1kq+eJrh0X5zIxUkFl1oyOw/fl1qDPHB6zxp0Ebd9fZcQP8Kq2cxRdHjcisI 3WPQ== X-Gm-Message-State: AOAM530p7YVcjjPGSAjb8KCQ3N7SYo5WH0K++/8bEnCJ9p7+B73P2hTf CVLNR4qOwVyiBxdMIsCBcyg= X-Google-Smtp-Source: ABdhPJwZB6LjTprMZwU2/pLMsLcOKXILgp2dWIm8lJ69t9iEHzDAwmju9I67y4dUy+fJWwnXK/BMzQ== X-Received: by 2002:a5d:58d8:: with SMTP id o24mr32666664wrf.288.1620135815664; Tue, 04 May 2021 06:43:35 -0700 (PDT) Received: from localhost ([137.220.125.106]) by smtp.gmail.com with ESMTPSA id o62sm2910761wmo.3.2021.05.04.06.43.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 May 2021 06:43:34 -0700 (PDT) Message-ID: <177ac3ccb8dbb0dbc3a18c6ce75c53b691b38d47.camel@debian.org> Subject: Re: Storing package metadata in ELF objects From: Luca Boccassi To: Mark Wielaard , Zbigniew =?UTF-8?Q?J=C4=99drzejewski-Szmek?= Cc: devel@lists.fedoraproject.org, systemd-devel@lists.freedesktop.org, debhelper@packages.debian.org, binutils@sourceware.org, Lennart Poettering , debian-dpkg@lists.debian.org, "elfutils-devel@sourceware.org" Date: Tue, 04 May 2021 14:43:32 +0100 In-Reply-To: References: <2b91ec1654b6c07cca2b5c113df772c85c0dd22c.camel@debian.org> <20210410184407.GC1503@in.waw.pl> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-uBZYqi7EQ2FwZlgFIR5X" User-Agent: Evolution 3.30.5-1.2 MIME-Version: 1.0 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: elfutils-devel@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Elfutils-devel mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 May 2021 13:43:38 -0000 --=-uBZYqi7EQ2FwZlgFIR5X Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, 2021-04-30 at 19:57 +0200, Mark Wielaard wrote: > Hi, >=20 > On Sat, 2021-04-10 at 18:44 +0000, Zbigniew J=C4=99drzejewski-Szmek wrote= : > > [I'm forwarding the mail from Luca who is not subscribed to fedora- > > devel] > > On Sat, Apr 10, 2021 at 01:38:31PM +0100, Luca Boccassi wrote: > > Cross-posting to the mailing lists of a few relevant projects. >=20 > Note that in this version of the email the [N] references in your email > don't seem to point anywhere. I found an older variant of the same > email which contained: >=20 > [0] https://github.com/systemd/systemd/issues/18433 > [1] https://systemd.io/COREDUMP_PACKAGE_METADATA/ > [2] https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_obj= ects > [3] https://github.com/systemd/package-notes Sorry about that! Must have messed up the copy&paste. > > After an initial discussion [0], recently we have been working on a new > > specification [0] to encode rich package-level metadata inside ELF > > objects, so that it can be included automatically in generated coredump > > files. The prototype to parse this in systemd-coredump and store the > > information in systemd-journal is ready for testing and merged > > upstream. We are now seeking further comments/opinions/suggestions, as > > we have a few months before the next release and thus there's plenty of > > time to make incompatible changes to the format and implementation, if > > required. > >=20 > > A proposal to use this by default for all packages built in Fedora 35 > > has been submitted [1]. > >=20 > > The Fedora Wiki and the systemd.io document have more details, but to > > make a long story short, a new .notes.package section with a JSON > > payload will be included in ELF objects, encoding various package- > > build-time information like distro name&version, package > > name&version, > > etc. >=20 > Is there a list of default keys (and their canonical spelling, upper- > lower-Camel_Case, etc.)? If there is, could we have a "debuginfod" key > with as value an URL pointing to the debuginfod server URL where the > embedded build-id executable, debuginfo and sources can be found? > https://sourceware.org/elfutils/Debuginfod.html The "Implementation" section of the spec lists the "main" fields: https://systemd.io/COREDUMP_PACKAGE_METADATA/ (source for that is https://github.com/systemd/systemd/blob/main/docs/CORED= UMP_PACKAGE_METADATA.md ) Would you like to send a PR to update it and add that field? > > To summarize from the discussion, the main reasons why we believe this > > is useful are as following: > >=20 > > 1) minimal containers: the rpm database is not installed in the > > containers. The information about build-ids needs to be stored > > externally, so package name information is not available immediately, > > but only after offline processing. The new note doesn't depend on the > > rpm db in any way. > >=20 > > 2) handling of a core from a container, where the container and host > > have different distros > >=20 > > 3) self-built and external packages: unless a lot of care is taken to > > keep access to the debuginfo packages, this information may be lost. > > The new note is available even if the repository metadata gets lost. > > Users can easily provide equivalent information in a format that makes > > sense in their own environment. It should work even when rpms and debs > > and other formats are mixed, e.g. during container image creation. > >=20 > > Other than in Fedora, we are already making the required code changes > > at Microsoft to use the same format&specification for internally-built > > binaries, and for tools that parse core files and logs. > >=20 > > Tools for RPM and DEB (debhelper) integration are also available [3]. > >=20 > > > --=20 > > > Kind regards, > > > Luca Boccassi --=20 Kind regards, Luca Boccassi --=-uBZYqi7EQ2FwZlgFIR5X Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEErCSqx93EIPGOymuRKGv37813JB4FAmCRT4QACgkQKGv37813 JB48XQ//b1rbj33tNAXXzGKxvCmRDT+PEw2r9PtOo7+TROH5G9JvSw+k3WiVy+PA uUfHNVRyZS+8rC3Rojh3qJjx5ym0QOpphqWYuAFqHcMbRwAq4QV12zgABuVrfevq dlVALLaIkZxISia5OiHEDYxtBWzrNOUROQRMxk2pM6vZohvEKE2egUfsISyKyhuf BVeIowyZBJ+DMTha2vUgpzmr6eu2iLY7IHPNSCNhbQTo5rAOUSn0xjwm+Aa8dGcW 0usQrLgZ6CZ5sq/28jYwU5veuiURoeKxksKYyuaP4/zl5BhdFRdJO8zmQTVn1YAT tCSRb6z7WJEAMoGpVjwVyKzEto2jMHOKMRxN7T6kaAuiC0ZcXueRd4a6NMxt64JF zrblWl3ksgxIlOAXus6uQQdOpti690PDDnu8gP8VbFdoBQAqI7t9xtm1e0sDoVCR YcRKdPcHG0k1gV6pVWOV7rz1qr3O7qgeMUVxLie3+04pcK/gwf2YjmwrTSe3pLuD 5+U3j1zjJ6qmhR+a94hmu4zJGwMY2CEHKMx1UsfKj4j2wQvopaJprRPZCUlXezt7 YrO9aB9tT5oGqssZM4G7ANNsjFaJbNte5EQnfr7WBmE4Pn8cjV6UCfubGBWeX8Bn u9NjqQCzKmgmHgKCb9V1RwIFHzGF69b0HO3ccVN/KGUJ2Zj1UwM= =0OcC -----END PGP SIGNATURE----- --=-uBZYqi7EQ2FwZlgFIR5X--