From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id AD0263858CD1 for ; Fri, 17 Nov 2023 14:53:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AD0263858CD1 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AD0263858CD1 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700232810; cv=none; b=YBZVq/zK3QS81ipbCruXD1x+ybBc/HG7KBzV4QHmn2ppYJY0cOAHck/buRKmmN1f66HZrUsm/V19MVlQ55b9LBupphnpkBq/LEZcn61gDk106je7NSL3Xu4xDKFlmcRoKntG+T+ShNhv5MGhn4841B8qt+QfTIPXO7myGpkTJJU= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1700232810; c=relaxed/simple; bh=nZcGoR+5oen13nIIxdPx3c/cNzlBhESH/vk3+7aeO2E=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=Lw2Uub45HNojJ9XyEltMFaXaYSKL9TR5Ah7EuaJE3CDxFoEhsDKzGNIJeOlkt50IX739hwnVKSMtyz5ZDvyZgUYfAsJ3mZm09IiPQmT5YNDH5ca/rs3yGElT0t8tHSgYcceHhGsPicuiRla5FzAsG4MPlWWjUDrTeVU4IKouR+I= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1700232808; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=d5jx7/z2tEzk28YpfhFlzF0wxXw8Y0ulZkk0NwOheys=; b=W9Jw5DXsCGdLWIwB9Ea6RKWoC5gonn6dYPt8vaNL+Etgms7WhD/380boajZv0cgmsLDiA6 9UtPGEnHbJfu1l9eXWoyzzgZaTrAaaIOQ7yR7jiKqnmA0XcnyNiPRinn7CophIvcmmtBmj mSr2s0JOaoIVHEXXeGKYNAIm+Iu9hOE= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-5-J-F3Ke-HNjKB_ie1H5Pgcg-1; Fri, 17 Nov 2023 09:53:25 -0500 X-MC-Unique: J-F3Ke-HNjKB_ie1H5Pgcg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2605A3C1AF4D; Fri, 17 Nov 2023 14:53:25 +0000 (UTC) Received: from redhat.com (unknown [10.22.8.8]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0884C40C6EB9; Fri, 17 Nov 2023 14:53:25 +0000 (UTC) Received: from [127.0.0.1] (helo=vm-rhel7) by redhat.com with esmtp (Exim 4.94.2) (envelope-from ) id 1r40Dj-0000eG-Ce; Fri, 17 Nov 2023 09:53:24 -0500 From: fche@redhat.com (Frank Ch. Eigler) To: Dodji Seketeli Cc: libabigail@sourceware.org, woodard@redhat.com Subject: Re: idea: abigail abixml archive Date: Fri, 17 Nov 2023 09:53:08 -0500 References: <20231115155306.GC15862@redhat.com> <87pm08y1qw.fsf@seketeli.org> Message-ID: <87zfzctorg.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain X-Spam-Status: No, score=-7.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Dodji Seketeli writes: >> The germ of the idea is that developers may need to know whether a >> binary they built or found is likely to be abi-compatible with a given >> distro / version. > > Yes, if my memory serves, Ben Woodard (in copy of this message) was the > first one I heard talking about this feature. Yes. > I am guessing this is useful for users who build a binary on a distro > and would like to know if they can run it on another distro without > using things like containers. Yes. > Am I right in expressing the user need here or am I missing something? And also to detect accidental ABI changes within one's own new binary or shared library, as compared with previous versions. Probably other uses too. > Today, even if you have access to the target distro, it's not practical > with the current tools to know if the binary is "ABI compatible" with > it. > > If I understand things correctly, you would need to: > > 1/ Get the ABI of the (transitive closure) set of dependencies of > the binary on its original distro. I call it orig-deps-abi. > 2/ Get the ABI of the set of dependencies of the binary on the > target distro. I call it target-deps-abi. > 3/ compare target-deps-abi against orig-deps-abi. Maybe? Is the transitive closure necessary? Heck, if we're just testing binary A against the remote shared libraries B C D, then the local libraries B C D may not be relevant at all. If so, then it's just comparing the local binary A vs. the directly dependentent remote libraries mentioned in DT_NEEDED. (To also compare local libraries B C D, one might rerun the tool with each of B C D as the left operand.) >[...] > OK. > > And so the tool that does [1-3] would just download get the archive of > source and target distributions and do the work locally. Correct? The target distributions, yes. Source distributions if relevant (not sure), but then again for source distribution it already has or could get the binaries+debuginfo (at the cost of downloading time). >> For storing this data, I envision overloading the libabigil git repo >> (or a new one) with storage of the abixml documents. > > I am guessing the new tool itself would be yet another libabigail tool > alongside the 6 we already have, so it would be in the libabigail git > repo. Sure. > But the distro ABI archives would better be hosted in another git repo > somewhere. And users might have their own private distro ABI repo > somewhere as they see fit. WDYT? Any random git repo, sure. > Heck, it could even just be in a directory tree served by whatever > transport protocol users would see fit. Git would be our preferred > choice, of course. Similarly to the way distro packages themselves are > organized today. Perhaps one could accept the file:// url configuration, and access it directly as though it were a git working tree checkout. But: >> To keep it dead simple, there could be one branch per /etc/os-release >> $ID/$VERSION_ID, one file per shared library in the distribution. For >> example, a fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file >> abidw produces could sit at >> >> repo git://sourceware.org/git/libabigail.git >> branch gitabixml/fedora/39/x86_64 >> file /usr/lib64/libc.so.6.xml > > I would even go further as to put the binary inside a subdirectory tree > with all that information, making it somewhat independent from being in git: hmm >> repo git://sourceware.org/git/distributions-abi.git >> file /fedora/39/x86_64/glibc/usr/lib64/libc.so.6.xml ^^^^^^^^^^^^^^^^^^##### While the ^^^^ first part can be known to the client, the #### package abbreviation may well not be. A binary just makes reference to sonames, which are only roughly file names, and definitely not package abbreviations. > Storing information about the package would be useful, for instance, to > handle conflicting packages that might have binaries with the same > path. In a way, that's a variant of having more than one version of the same binary in the history of the same distro. >>[...] >> Maybe abidw version tags would be useful to add. > > I am not sure what you mean by abidw version tag. (I meant the "abidw --version" string to identify the producer.) > A possible way to handle this in a way that is not dependent on Git > would be to store the originating n-v-r in the abixml directly. The > tool that emits the abixml (from the original package) would be able to > do that. Right, that's possible. But compared to a dead filesystem representation, git lets us store multiple objects with the same name, and lets the client choose which one to use (by traversing a branch history by time or metadata hint in the commit or the actual payload abixml itself). In a dead filesystem, such alternative/history operation would have to be done by materializing the history/variant axis somewhere in the path ... and clients would have to glob over that component. (Adding "glibc" wouldn't be enough. n-v-r-a at the least.) >> [...] >> and rerun that occasionally as updates flow down from the distro. >> This could be done on a single beefy box running containers with >> different distros. > > Yes, I like the idea of having something incremental like this. Right. > Also, we need to add a mode to libabigail to let it expect debuginfod > to find the debuginfo because today, it expects the user to provide > the debug info location in cases where it's not already installed on > the system. [...] Are you sure? I run libabigail tools without any manual -d type settings or debuginfod overrides, and it already just works, via elfutils' builtin debuginfod support. - FChE