idea: abigail abixml archive

idea: abigail abixml archive

[Frank Ch. Eigler]

Hi -

I'd love some feedback about the following idea, related to using
libabigail to assemble a crowdsourced database of abixml files for
linux distros.

The germ of the idea is that developers may need to know whether a
binary they built or found is likely to be abi-compatible with a given
distro / version.  This is possible today by downloading the target
distro binaries and running libabigail locally against them, or using
front-end scripts like fedabipkgdiff that do the downloading first.
But this is a pain if one wants to compare against a range of versions
or foreign distros.

So the idea is instead to let people use an public archive of abixml
artifacts instead of the binaries.  The abixml files are relatively
tiny, barely-ever changing, and should be an effective proxy for the
real binaries.  It's just a small matter of (a) storing, (b)
using, and (c) collecting it.

--------------------

For storing this data, I envision overloading the libabigil git repo
(or a new one) with storage of the abixml documents.  To keep it dead
simple, there could be one branch per /etc/os-release $ID/$VERSION_ID,
one file per shared library in the distribution.  For example, a
fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file abidw produces
could sit at

   repo  git://sourceware.org/git/libabigail.git 
 branch  gitabixml/fedora/39/x86_64
   file  /usr/lib64/libc.so.6.xml

(Symlinks in the distro fs could be represented as symlinks in git.)

Updates to the distro package of course happen.  It seems natural to
update the abixml file for the affected file(s) right there in place.

Since it may sometimes be desirable to track what package version
(e.g. rpm n-v-r) is associated with the abixml data of a given
version, we could use stylized records in the git commit text (or a
git note, or maybe a tag).  That would mean one git commit per updated
package, with metadata message like:

  Package: glibc-2.38-7.fc39.x86_64

Maybe abidw version tags would be useful to add.

--------------------

For using this data, I envision abidiff / abicompat taking a new form
for its right operand.  It could be a git url identifying the distro
branch or tag.  libabigail would fetch the corresponding file.xml
within that.  Simplify/default the heck out of it for ease of use:

  export $BRANCH=fedora/39/x86_64
  abicompat /bin/myprogram gitabixml:$BRANCH

(Where "gitabixml:" could instruct the tool to look at the sourceware
libabigail git / gitweb / cgit server.  Let users specify different or
private git servers via environment variables or something.

--------------------

For collecting this data, I envision writing some distro-specific
scripts, kind of like fedabipkgdiff, being run by contributors or
ourselves.  One flavour could run in operational installed distros,
doing the equivalent of

    find $PATHS -name '*.so.*' | while read lib; do
       # or filter with elfclassify 
       package=`rpm -qf "$lib"` 
       abidw "$lib" | (cd $gitrepo/`dirname $lib`; cat > "$lib.xml")
       (cd $gitrepo; git commit -m"Package: $package" "$lib.xml")
    done

and rerun that occasionally as updates flow down from the distro.
This could be done on a single beefy box running containers with
different distros.

Another flavour could be to take a set of RPM/etc. archives on a
filesystem (or an ISO image), incrementally decompress them, run abidw
on the individual files, and similarly construct the git repo of
abixml files.  (This is kind of like how debuginfod produces indexes
from a bunch of RPMs.)

No matter how the local git repo is populated, each branch describing
a data contributor's distro could be pushed to the central one,
bringing that one up to date.  Patches representing updates could be
emailed too, but no one will want to read/review that stuff.  We'd
probably need a trusted pool of contributors who can just commit to
areas of the central git repo.  Secured with gitsigur of course. :-)

The central repo could be built up entirely gradually.  If some
libraries were omitted from initial commits for a distro, a later
contribution could fill in the gaps.

--------------------

OK, how reasonable does all this sound? 


- FChE

Re: idea: abigail abixml archive

[Dodji Seketeli]

Hello,

"Frank Ch. Eigler" <fche@redhat.com> a écrit:

> Hi -
>
> I'd love some feedback about the following idea, related to using
> libabigail to assemble a crowdsourced database of abixml files for
> linux distros.
>
> The germ of the idea is that developers may need to know whether a
> binary they built or found is likely to be abi-compatible with a given
> distro / version.

Yes, if my memory serves, Ben Woodard (in copy of this message) was the
first one I heard talking about this feature.

I am guessing this is useful for users who build a binary on a distro
and would like to know if they can run it on another distro without
using things like containers.

Am I right in expressing the user need here or am I missing something?

> This is possible today by downloading the target distro binaries and
> running libabigail locally against them, or using front-end scripts
> like fedabipkgdiff that do the downloading first.

Today, even if you have access to the target distro, it's not practical
with the current tools to know if the binary is "ABI compatible" with
it.

If I understand things correctly, you would need to:

    1/ Get the ABI of the (transitive closure) set of dependencies of
    the binary on its original distro.  I call it orig-deps-abi.

    2/ Get the ABI of the set of dependencies of the binary on the
    target distro.  I call it target-deps-abi.

    3/ compare target-deps-abi against orig-deps-abi.

abicompat, for instance, hasn't been designed for that.  We would need
to either extend it to support 1-3 or come up with a new tool for that.

> But this is a pain if one wants to compare against a range of versions
> or foreign distros.

Indeed.

> So the idea is instead to let people use an public archive of abixml
> artifacts instead of the binaries.  The abixml files are relatively
> tiny, barely-ever changing, and should be an effective proxy for the
> real binaries.  It's just a small matter of (a) storing, (b)
> using, and (c) collecting it.

OK.

And so the tool that does [1-3] would just download get the archive of
source and target distributions and do the work locally.  Correct?

> --------------------
>
> For storing this data, I envision overloading the libabigil git repo
> (or a new one) with storage of the abixml documents.

I am guessing the new tool itself would be yet another libabigail tool
alongside the 6 we already have, so it would be in the libabigail git
repo.

But the distro ABI archives would better be hosted in another git repo
somewhere.  And users might have their own private distro ABI repo
somewhere as they see fit.  WDYT?

Heck, it could even just be in a directory tree served by whatever
transport protocol users would see fit.  Git would be our preferred
choice, of course.  Similarly to the way distro packages themselves are
organized today.

>  To keep it dead simple, there could be one branch per /etc/os-release
> $ID/$VERSION_ID, one file per shared library in the distribution.  For
> example, a fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file
> abidw produces could sit at
>
>    repo  git://sourceware.org/git/libabigail.git 
>  branch  gitabixml/fedora/39/x86_64
>    file  /usr/lib64/libc.so.6.xml

I would even go further as to put the binary inside a subdirectory tree
with all that information, making it somewhat independent from being in git:

>    repo  git://sourceware.org/git/distributions-abi.git
>    file  /fedora/39/x86_64/glibc/usr/lib64/libc.so.6.xml


Storing information about the package would be useful, for instance, to
handle conflicting packages that might have binaries with the same
path.

> (Symlinks in the distro fs could be represented as symlinks in git.)

ACK.


> Updates to the distro package of course happen.  It seems natural to
> update the abixml file for the affected file(s) right there in place.

Yes. Updating an abixml file would just be an overwrite.  We would not
need to handle merges etc.

> Since it may sometimes be desirable to track what package version
> (e.g. rpm n-v-r) is associated with the abixml data of a given
> version, we could use stylized records in the git commit text (or a
> git note, or maybe a tag).  That would mean one git commit per updated
> package, with metadata message like:
>
>   Package: glibc-2.38-7.fc39.x86_64
>
> Maybe abidw version tags would be useful to add.

I am not sure what you mean by abidw version tag.

A possible way to handle this in a way that is not dependent on Git
would be to store the originating n-v-r in the abixml directly.  The
tool that emits the abixml (from the original package) would be able to
do that.

> --------------------
>
> For using this data, I envision abidiff / abicompat taking a new form
> for its right operand.  It could be a git url identifying the distro
> branch or tag.  libabigail would fetch the corresponding file.xml
> within that.  Simplify/default the heck out of it for ease of use:
>
>   export $BRANCH=fedora/39/x86_64
>   abicompat /bin/myprogram gitabixml:$BRANCH
>
> (Where "gitabixml:" could instruct the tool to look at the sourceware
> libabigail git / gitweb / cgit server.  Let users specify different or
> private git servers via environment variables or something.

Yes, something like that.  I guess the specific will depend on what we
end up settling on for the above (and below).


> --------------------
>
> For collecting this data, I envision writing some distro-specific
> scripts, kind of like fedabipkgdiff, being run by contributors or
> ourselves.  One flavour could run in operational installed distros,
> doing the equivalent of
>
>     find $PATHS -name '*.so.*' | while read lib; do
>        # or filter with elfclassify 
>        package=`rpm -qf "$lib"` 
>        abidw "$lib" | (cd $gitrepo/`dirname $lib`; cat > "$lib.xml")
>        (cd $gitrepo; git commit -m"Package: $package" "$lib.xml")
>     done
>
> and rerun that occasionally as updates flow down from the distro.
> This could be done on a single beefy box running containers with
> different distros.

Yes, I like the idea of having something incremental like this.

There would probably be some tweaks added to abidw to let it add a
"version string" (the N-V-R mentioned earlier) to the abidw, but these
are details.  Also, we need to add a mode to libabigail to let it expect
debuginfod to find the debuginfo because today, it expects the user to
provide the debug info location in cases where it's not already
installed on the system.  Again, this is a detail but it's going to
matter as soon as we the rubber hits the road.

> Another flavour could be to take a set of RPM/etc. archives on a
> filesystem (or an ISO image), incrementally decompress them, run abidw
> on the individual files, and similarly construct the git repo of
> abixml files.  (This is kind of like how debuginfod produces indexes
> from a bunch of RPMs.)

ACK.
>
> No matter how the local git repo is populated, each branch describing
> a data contributor's distro could be pushed to the central one,
> bringing that one up to date.  Patches representing updates could be
> emailed too, but no one will want to read/review that stuff.  We'd
> probably need a trusted pool of contributors who can just commit to
> areas of the central git repo.  Secured with gitsigur of course. :-)
>
> The central repo could be built up entirely gradually.  If some
> libraries were omitted from initial commits for a distro, a later
> contribution could fill in the gaps.

ACK.

Thank you for putting this thoughts together.  The idea is getting much
less abstract in my mind now.

Cheers,

-- 
		Dodji

Re: idea: abigail abixml archive

[Frank Ch. Eigler]

Dodji Seketeli <dodji@seketeli.org> writes:

>> The germ of the idea is that developers may need to know whether a
>> binary they built or found is likely to be abi-compatible with a given
>> distro / version.
>
> Yes, if my memory serves, Ben Woodard (in copy of this message) was the
> first one I heard talking about this feature.

Yes.

> I am guessing this is useful for users who build a binary on a distro
> and would like to know if they can run it on another distro without
> using things like containers.

Yes.

> Am I right in expressing the user need here or am I missing something?

And also to detect accidental ABI changes within one's own new binary or
shared library, as compared with previous versions.  Probably other uses
too.


> Today, even if you have access to the target distro, it's not practical
> with the current tools to know if the binary is "ABI compatible" with
> it.
>
> If I understand things correctly, you would need to:
>
>     1/ Get the ABI of the (transitive closure) set of dependencies of
>     the binary on its original distro.  I call it orig-deps-abi.
>     2/ Get the ABI of the set of dependencies of the binary on the
>     target distro.  I call it target-deps-abi.
>     3/ compare target-deps-abi against orig-deps-abi.

Maybe?  Is the transitive closure necessary?  Heck, if we're just
testing binary A against the remote shared libraries B C D, then the
local libraries B C D may not be relevant at all.  If so, then it's just
comparing the local binary A vs. the directly dependentent remote
libraries mentioned in DT_NEEDED.

(To also compare local libraries B C D, one might rerun the tool with
each of B C D as the left operand.)


>[...]
> OK.
>
> And so the tool that does [1-3] would just download get the archive of
> source and target distributions and do the work locally.  Correct?

The target distributions, yes.  Source distributions if relevant (not
sure), but then again for source distribution it already has or could
get the binaries+debuginfo (at the cost of downloading time).


>> For storing this data, I envision overloading the libabigil git repo
>> (or a new one) with storage of the abixml documents.
>
> I am guessing the new tool itself would be yet another libabigail tool
> alongside the 6 we already have, so it would be in the libabigail git
> repo.

Sure.

> But the distro ABI archives would better be hosted in another git repo
> somewhere.  And users might have their own private distro ABI repo
> somewhere as they see fit.  WDYT?

Any random git repo, sure.

> Heck, it could even just be in a directory tree served by whatever
> transport protocol users would see fit.  Git would be our preferred
> choice, of course.  Similarly to the way distro packages themselves are
> organized today.

Perhaps one could accept the file:// url configuration, and access it
directly as though it were a git working tree checkout.  But:


>> To keep it dead simple, there could be one branch per /etc/os-release
>> $ID/$VERSION_ID, one file per shared library in the distribution.  For
>> example, a fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file
>> abidw produces could sit at
>>
>>    repo  git://sourceware.org/git/libabigail.git 
>>  branch  gitabixml/fedora/39/x86_64
>>    file  /usr/lib64/libc.so.6.xml
>
> I would even go further as to put the binary inside a subdirectory tree
> with all that information, making it somewhat independent from being in git:

hmm

>>    repo  git://sourceware.org/git/distributions-abi.git
>>    file  /fedora/39/x86_64/glibc/usr/lib64/libc.so.6.xml
            ^^^^^^^^^^^^^^^^^^#####

While the ^^^^ first part can be known to the client, the #### package
abbreviation may well not be.  A binary just makes reference to sonames,
which are only roughly file names, and definitely not package
abbreviations.


> Storing information about the package would be useful, for instance, to
> handle conflicting packages that might have binaries with the same
> path.

In a way, that's a variant of having more than one version of the same
binary in the history of the same distro.


>>[...]
>> Maybe abidw version tags would be useful to add.
>
> I am not sure what you mean by abidw version tag.

(I meant the "abidw --version" string to identify the producer.)


> A possible way to handle this in a way that is not dependent on Git
> would be to store the originating n-v-r in the abixml directly.  The
> tool that emits the abixml (from the original package) would be able to
> do that.

Right, that's possible.  But compared to a dead filesystem
representation, git lets us store multiple objects with the same name,
and lets the client choose which one to use (by traversing a branch
history by time or metadata hint in the commit or the actual payload
abixml itself).  In a dead filesystem, such alternative/history
operation would have to be done by materializing the history/variant
axis somewhere in the path ... and clients would have to glob over that
component.  (Adding "glibc" wouldn't be enough.  n-v-r-a at the least.)


>> [...]
>> and rerun that occasionally as updates flow down from the distro.
>> This could be done on a single beefy box running containers with
>> different distros.
>
> Yes, I like the idea of having something incremental like this.

Right.


> Also, we need to add a mode to libabigail to let it expect debuginfod
> to find the debuginfo because today, it expects the user to provide
> the debug info location in cases where it's not already installed on
> the system.  [...]

Are you sure?  I run libabigail tools without any manual -d type
settings or debuginfod overrides, and it already just works, via
elfutils' builtin debuginfod support.


- FChE

Re: idea: abigail abixml archive

[Dodji Seketeli]

Hello,

[...]

Dodji Seketeli <dodji@seketeli.org> writes:

>> Today, even if you have access to the target distro, it's not practical
>> with the current tools to know if the binary is "ABI compatible" with
>> it.
>>
>> If I understand things correctly, you would need to:
>>
>>     1/ Get the ABI of the (transitive closure) set of dependencies of
>>     the binary on its original distro.  I call it orig-deps-abi.
>>     2/ Get the ABI of the set of dependencies of the binary on the
>>     target distro.  I call it target-deps-abi.
>>     3/ compare target-deps-abi against orig-deps-abi.

fche@redhat.com (Frank Ch. Eigler) a écrit:

> Heck, if we're just testing binary A against the remote shared
> libraries B C D, then the local libraries B C D may not be relevant at
> all.

I think it all boils down to what we mean by "testing binary A against
its dependency".

When we want to know if a binary A is compatible with a library B, we
consider functions & global variables undefined in A and defined (and
exported) in B.  The DWARF for the types of the undefined functions &
variables are present in A.  These are the types "expected" by A.  In B,
there is going to be same types, because they come with the definition
of the exported functions & variables that are expected by A.  So these
types are "provided" by B.

We can compare the types expected by A to their counterpart provided by
B and if they are equal, then A is compatible with B, as far as these
functions & variables and their types are concerned.

This is basically what abicompat does.

But then there is a case of incompatibility that we can't catch by
looking at just A and B.  That is, if a function that was previously
defined in B (and is needed by A) is removed from B.  In that case, we
need to know if that function was moved to another dependency of A.  It
only is if the function is removed from all the dependencies of A that
we can know for sure that the function is missing, and thus, that A is
facing an incompatible change in that regard.

>  Is the transitive closure necessary?

I think the transitive closure would be necessary for the case I
explained above, because a removed function could have been moved to any
other dependant library.  Or am I missing something?

> If so, then it's just comparing the local binary A vs. the
> directly dependentent remote libraries mentioned in DT_NEEDED.

If you only want to consider the dependant remote dependencies, then
that would amount to doing what abicompat does today, more or less.  I
think that could work too (instead of going through [1-3]).  The only
caveat I would have is the transitive closure one.

[...]


fche@redhat.com (Frank Ch. Eigler) a écrit:

>>> To keep it dead simple, there could be one branch per /etc/os-release
>>> $ID/$VERSION_ID, one file per shared library in the distribution.  For
>>> example, a fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file
>>> abidw produces could sit at
>>>
>>>    repo  git://sourceware.org/git/libabigail.git 
>>>  branch  gitabixml/fedora/39/x86_64
>>>    file  /usr/lib64/libc.so.6.xml

Dodji Seketeli <dodji@seketeli.org> writes:

>>
>> I would even go further as to put the binary inside a subdirectory tree
>> with all that information, making it somewhat independent from being in git:

fche@redhat.com (Frank Ch. Eigler) a écrit:

> hmm

>>>    repo  git://sourceware.org/git/distributions-abi.git
>>>    file  /fedora/39/x86_64/glibc/usr/lib64/libc.so.6.xml
>             ^^^^^^^^^^^^^^^^^^#####
>
> While the ^^^^ first part can be known to the client, the #### package
> abbreviation may well not be.  A binary just makes reference to sonames,
> which are only roughly file names, and definitely not package
> abbreviations.

Indeed, not all binary present on the user's system come from a package.
And the ABI of those binaries might indeed end up in the repository.
Point taken.

OK then, I take the package thing back.

Let's just store the thing in Git as you proposed.

[...]


>> Also, we need to add a mode to libabigail to let it expect debuginfod
>> to find the debuginfo because today, it expects the user to provide
>> the debug info location in cases where it's not already installed on
>> the system.  [...]
>
> Are you sure?  I run libabigail tools without any manual -d type
> settings or debuginfod overrides, and it already just works, via
> elfutils' builtin debuginfod support.
>

Interesting.  I must double check.  I must have broken something locally
then.  I am pretty sure to remember I've had to add the --d{1,2} to
abipkgdiff otherwise it wasn't finding debuginfo.  But I need to double
check.  It it works that's great :-)

Cheers,

-- 
		Dodji

Re: idea: abigail abixml archive

[Giuliano Procida]

Hi.

On Wed, 15 Nov 2023 at 15:53, Frank Ch. Eigler <fche@redhat.com> wrote:
>
> Hi -
>
> I'd love some feedback about the following idea, related to using
> libabigail to assemble a crowdsourced database of abixml files for
> linux distros.
>
> The germ of the idea is that developers may need to know whether a
> binary they built or found is likely to be abi-compatible with a given
> distro / version.  This is possible today by downloading the target
> distro binaries and running libabigail locally against them, or using
> front-end scripts like fedabipkgdiff that do the downloading first.
> But this is a pain if one wants to compare against a range of versions
> or foreign distros.
>
> So the idea is instead to let people use an public archive of abixml
> artifacts instead of the binaries.  The abixml files are relatively
> tiny, barely-ever changing, and should be an effective proxy for the
> real binaries.  It's just a small matter of (a) storing, (b)
> using, and (c) collecting it.
>
> --------------------
>
> For storing this data, I envision overloading the libabigil git repo
> (or a new one) with storage of the abixml documents.  To keep it dead
> simple, there could be one branch per /etc/os-release $ID/$VERSION_ID,
> one file per shared library in the distribution.  For example, a
> fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file abidw produces
> could sit at
>
>    repo  git://sourceware.org/git/libabigail.git
>  branch  gitabixml/fedora/39/x86_64
>    file  /usr/lib64/libc.so.6.xml
>
> (Symlinks in the distro fs could be represented as symlinks in git.)
>
> Updates to the distro package of course happen.  It seems natural to
> update the abixml file for the affected file(s) right there in place.
>
> Since it may sometimes be desirable to track what package version
> (e.g. rpm n-v-r) is associated with the abixml data of a given
> version, we could use stylized records in the git commit text (or a
> git note, or maybe a tag).  That would mean one git commit per updated
> package, with metadata message like:
>
>   Package: glibc-2.38-7.fc39.x86_64
>
> Maybe abidw version tags would be useful to add.
>
> --------------------
>
> For using this data, I envision abidiff / abicompat taking a new form
> for its right operand.  It could be a git url identifying the distro
> branch or tag.  libabigail would fetch the corresponding file.xml
> within that.  Simplify/default the heck out of it for ease of use:
>
>   export $BRANCH=fedora/39/x86_64
>   abicompat /bin/myprogram gitabixml:$BRANCH
>
> (Where "gitabixml:" could instruct the tool to look at the sourceware
> libabigail git / gitweb / cgit server.  Let users specify different or
> private git servers via environment variables or something.
>
> --------------------
>
> For collecting this data, I envision writing some distro-specific
> scripts, kind of like fedabipkgdiff, being run by contributors or
> ourselves.  One flavour could run in operational installed distros,
> doing the equivalent of
>
>     find $PATHS -name '*.so.*' | while read lib; do
>        # or filter with elfclassify
>        package=`rpm -qf "$lib"`
>        abidw "$lib" | (cd $gitrepo/`dirname $lib`; cat > "$lib.xml")
>        (cd $gitrepo; git commit -m"Package: $package" "$lib.xml")
>     done
>
> and rerun that occasionally as updates flow down from the distro.
> This could be done on a single beefy box running containers with
> different distros.
>
> Another flavour could be to take a set of RPM/etc. archives on a
> filesystem (or an ISO image), incrementally decompress them, run abidw
> on the individual files, and similarly construct the git repo of
> abixml files.  (This is kind of like how debuginfod produces indexes
> from a bunch of RPMs.)
>
> No matter how the local git repo is populated, each branch describing
> a data contributor's distro could be pushed to the central one,
> bringing that one up to date.  Patches representing updates could be
> emailed too, but no one will want to read/review that stuff.  We'd
> probably need a trusted pool of contributors who can just commit to
> areas of the central git repo.  Secured with gitsigur of course. :-)
>
> The central repo could be built up entirely gradually.  If some
> libraries were omitted from initial commits for a distro, a later
> contribution could fill in the gaps.
>
> --------------------
>
> OK, how reasonable does all this sound?

This sounds like an interesting project, but you can go further.

Starting at the level of a single binary with a single .so. The
questions we want answered are:

1. Will all undefined symbols resolve successfully (otherwise
reporting missing symbols)?
2. Are the types of the resolved symbols compatible (otherwise
reporting differences)?
3. Do we have ABI representations that let us answer 1. and 2. without
having some binaries to hand? (Not yet.)

Neither libabigail nor STG emit undefined symbols in their ABI
representations (yet), so answering 1. currently requires having
binaries and debug information to hand.

Now build this up to multiple binaries, SONAME, bundled and unbundled
shared objects, library dependencies (ELF needs), multiple
distributions, packages, versions and architectures, supporting
link-loaded plugins with dlopen etc.

Having a full database of existing libraries would allow compatibility
for a freshly-compiled binary (or package) to be checked (idea due to
Matthias Männich) or dependency hell to be explored without actually
installing any packages.

Similarly, if binaries (and their packages) are in the database,
questions about library upgrades could be answered.

Giuliano.

>
> - FChE
>

Re: idea: abigail abixml archive

[Ben Woodard]

[-- Attachment #1: Type: text/plain, Size: 7663 bytes --]


On 11/15/23 07:53, Frank Ch. Eigler wrote:
> Hi -
>
> I'd love some feedback about the following idea, related to using
> libabigail to assemble a crowdsourced database of abixml files for
> linux distros.
>
> The germ of the idea is that developers may need to know whether a
> binary they built or found is likely to be abi-compatible with a given
> distro / version.  This is possible today by downloading the target
> distro binaries and running libabigail locally against them, or using
> front-end scripts like fedabipkgdiff that do the downloading first.
> But this is a pain if one wants to compare against a range of versions
> or foreign distros.

The other big use is "do I need to rebuild my binary for this new distro?"

This can be either a new Y release in RHEL X.Y where you have built for 
RHEL 9.2 and you want to make damn well sure that your program is 
compatible with RHEL9.3. Yes we make ABI stability guarantees but that 
only covers a portion of the distro. Or it could be you have your 
program built for Fedora 38 and you want to know if you need to rebuild 
it for Fedora 39 or if it happens to be ABI compatible.

The other use case is within the distro. You are a packager and you just 
applied a patch. You want to make sure that you didn't break ABI with 
the patch before you release it onto the world.

>
> So the idea is instead to let people use an public archive of abixml
> artifacts instead of the binaries.  The abixml files are relatively
> tiny, barely-ever changing, and should be an effective proxy for the
> real binaries.  It's just a small matter of (a) storing, (b)
> using, and (c) collecting it.

a) generating
b) storing
c) using
d) collecting

I add a) generating because as I have shown in the past there are many 
cases where libabigail needs to have some bugs fixed to be able to 
generate abixml files for particular packages.

> --------------------
>
> For storing this data, I envision overloading the libabigil git repo
> (or a new one) with storage of the abixml documents.  To keep it dead
> simple, there could be one branch per /etc/os-release $ID/$VERSION_ID,
> one file per shared library in the distribution.  For example, a
> fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file abidw produces
> could sit at
>
>     repo  git://sourceware.org/git/libabigail.git
>   branch  gitabixml/fedora/39/x86_64
>     file  /usr/lib64/libc.so.6.xml
>
> (Symlinks in the distro fs could be represented as symlinks in git.)
>
> Updates to the distro package of course happen.  It seems natural to
> update the abixml file for the affected file(s) right there in place.
>
> Since it may sometimes be desirable to track what package version
> (e.g. rpm n-v-r) is associated with the abixml data of a given
> version, we could use stylized records in the git commit text (or a
> git note, or maybe a tag).  That would mean one git commit per updated
> package, with metadata message like:
>
>    Package: glibc-2.38-7.fc39.x86_64
>
> Maybe abidw version tags would be useful to add.

Yeah, one challenge is going to be when for whatever reason dodji needs 
to update the abixml file format. In theory it shouldn't happen that 
often but when it does, it will be a HUGE churn in the git repo. I would 
suggest to mitigate that problem, there should be a abixml version in 
the pathname.

I know that one of the things that we need in the not too distant future 
is to track ABI on inline functions in C++ and so I expect that will 
need a AIBXML format change when it happens.

>
> --------------------
>
> For using this data, I envision abidiff / abicompat taking a new form
> for its right operand.  It could be a git url identifying the distro
> branch or tag.  libabigail would fetch the corresponding file.xml
> within that.  Simplify/default the heck out of it for ease of use:
>
>    export $BRANCH=fedora/39/x86_64
>    abicompat /bin/myprogram gitabixml:$BRANCH
>
> (Where "gitabixml:" could instruct the tool to look at the sourceware
> libabigail git / gitweb / cgit server.  Let users specify different or
> private git servers via environment variables or something.

As Dodji pointed out below abicompat doesn't currently handle 
dependencies very well. This has been a long term disagreement between 
us. To try to simplify (possibly oversimplify) the disagreement. Dodji 
believes that abicompat does one thing and that is test the forward ABI 
dependencies from the program to a particular library. Very Unixy do one 
thing. I disagree with this philosophy and all of my discussions with 
users indicate that they misinterpret what abicompat actually does 
thinking that it does more than it does. I'm a strong proponent of 
making abicompat do what the name implies and what users believe it does 
and that is actually check that the library is compatible with the 
program specified on the command line. This references another design 
principle: The principle of least suprise.

Thus IMHO abicompat should:

1) examine both forward and back dependencies. i.e. if a library makes 
use of a function or variable in the program itself then abicompat 
should consider that an ABI mismatch.
2) it should include all the dependencies of the program when 
considering ABI compatibility. Yes this could be made into a script but 
it ends up being a very inefficient and complicated script. I've written 
it two different times and people complain about the performance. The 
problem is it has to load the abixml over and over rather than keeping 
it in memory when doing the comparison. On complicated programs with 
lots of dependencies, this takes considerable time.


>
> --------------------
>
> For collecting this data, I envision writing some distro-specific
> scripts, kind of like fedabipkgdiff, being run by contributors or
> ourselves.  One flavour could run in operational installed distros,
> doing the equivalent of
>
>      find $PATHS -name '*.so.*' | while read lib; do
>         # or filter with elfclassify
>         package=`rpm -qf "$lib"`
>         abidw "$lib" | (cd $gitrepo/`dirname $lib`; cat > "$lib.xml")
>         (cd $gitrepo; git commit -m"Package: $package" "$lib.xml")
>      done

A way to gather this easily with the current framework would be to allow 
BRANCH to refer to a local clone of the abirepo and then let 
fedabipkgdiff stick its temporary files in there so that it could easily 
be committed.  That way when I do one of my tests which does a self 
comparison for every RPM, it generates the full data set.

> and rerun that occasionally as updates flow down from the distro.
> This could be done on a single beefy box running containers with
> different distros.
>
> Another flavour could be to take a set of RPM/etc. archives on a
> filesystem (or an ISO image), incrementally decompress them, run abidw
> on the individual files, and similarly construct the git repo of
> abixml files.  (This is kind of like how debuginfod produces indexes
> from a bunch of RPMs.)
>
> No matter how the local git repo is populated, each branch describing
> a data contributor's distro could be pushed to the central one,
> bringing that one up to date.  Patches representing updates could be
> emailed too, but no one will want to read/review that stuff.  We'd
> probably need a trusted pool of contributors who can just commit to
> areas of the central git repo.  Secured with gitsigur of course. :-)
>
> The central repo could be built up entirely gradually.  If some
> libraries were omitted from initial commits for a distro, a later
> contribution could fill in the gaps.
>
> --------------------
>
> OK, how reasonable does all this sound?
>
>
> - FChE
>

Re: idea: abigail abixml archive

[Ben Woodard]

On 11/17/23 04:59, Dodji Seketeli wrote:
> Hello,
>
> "Frank Ch. Eigler" <fche@redhat.com> a écrit:
>
>> Hi -
>>
>> I'd love some feedback about the following idea, related to using
>> libabigail to assemble a crowdsourced database of abixml files for
>> linux distros.
>>
>> The germ of the idea is that developers may need to know whether a
>> binary they built or found is likely to be abi-compatible with a given
>> distro / version.
> Yes, if my memory serves, Ben Woodard (in copy of this message) was the
> first one I heard talking about this feature.
>
> I am guessing this is useful for users who build a binary on a distro
> and would like to know if they can run it on another distro without
> using things like containers.
>
> Am I right in expressing the user need here or am I missing something?
>
>> This is possible today by downloading the target distro binaries and
>> running libabigail locally against them, or using front-end scripts
>> like fedabipkgdiff that do the downloading first.
> Today, even if you have access to the target distro, it's not practical
> with the current tools to know if the binary is "ABI compatible" with
> it.
>
> If I understand things correctly, you would need to:
>
>      1/ Get the ABI of the (transitive closure) set of dependencies of
>      the binary on its original distro.  I call it orig-deps-abi.
>
>      2/ Get the ABI of the set of dependencies of the binary on the
>      target distro.  I call it target-deps-abi.
>
>      3/ compare target-deps-abi against orig-deps-abi.
>
> abicompat, for instance, hasn't been designed for that.  We would need
> to either extend it to support 1-3 or come up with a new tool for that.

I continue to believe that the current design of abicompat is not 
sufficient for what users want to do. The name is misleading and what it 
currently accomplishes not only doesn't match user's expectations given 
its name, What it does accomplish is not very useful from a user's 
perspective. We have literally not been able to come up with an actual 
use for abicompat as it now exists without putting a ton of scripting 
around it. Therefore I strongly believe that it should be extended to 
actually compute the forward and backward ABI dependencies of the 
program and its library dependencies. This is what users  expect 
abicompat to do before they are educated on what it actually does and 
this is what users need it to do.

>
>> But this is a pain if one wants to compare against a range of versions
>> or foreign distros.
> Indeed.
>
>> So the idea is instead to let people use an public archive of abixml
>> artifacts instead of the binaries.  The abixml files are relatively
>> tiny, barely-ever changing, and should be an effective proxy for the
>> real binaries.  It's just a small matter of (a) storing, (b)
>> using, and (c) collecting it.
> OK.
>
> And so the tool that does [1-3] would just download get the archive of
> source and target distributions and do the work locally.  Correct?
>
>> --------------------
>>
>> For storing this data, I envision overloading the libabigil git repo
>> (or a new one) with storage of the abixml documents.
> I am guessing the new tool itself would be yet another libabigail tool
> alongside the 6 we already have, so it would be in the libabigail git
> repo.
>
> But the distro ABI archives would better be hosted in another git repo
> somewhere.  And users might have their own private distro ABI repo
> somewhere as they see fit.  WDYT?
>
> Heck, it could even just be in a directory tree served by whatever
> transport protocol users would see fit.  Git would be our preferred
> choice, of course.  Similarly to the way distro packages themselves are
> organized today.
>
>>   To keep it dead simple, there could be one branch per /etc/os-release
>> $ID/$VERSION_ID, one file per shared library in the distribution.  For
>> example, a fedora-39-x86-64 copy of /usr/lib64/libc.so.6, the file
>> abidw produces could sit at
>>
>>     repo  git://sourceware.org/git/libabigail.git
>>   branch  gitabixml/fedora/39/x86_64
>>     file  /usr/lib64/libc.so.6.xml
> I would even go further as to put the binary inside a subdirectory tree
> with all that information, making it somewhat independent from being in git:
>
>>     repo  git://sourceware.org/git/distributions-abi.git
>>     file  /fedora/39/x86_64/glibc/usr/lib64/libc.so.6.xml
>
> Storing information about the package would be useful, for instance, to
> handle conflicting packages that might have binaries with the same
> path.
>
>> (Symlinks in the distro fs could be represented as symlinks in git.)
> ACK.
>
>
>> Updates to the distro package of course happen.  It seems natural to
>> update the abixml file for the affected file(s) right there in place.
> Yes. Updating an abixml file would just be an overwrite.  We would not
> need to handle merges etc.
>
>> Since it may sometimes be desirable to track what package version
>> (e.g. rpm n-v-r) is associated with the abixml data of a given
>> version, we could use stylized records in the git commit text (or a
>> git note, or maybe a tag).  That would mean one git commit per updated
>> package, with metadata message like:
>>
>>    Package: glibc-2.38-7.fc39.x86_64
>>
>> Maybe abidw version tags would be useful to add.
> I am not sure what you mean by abidw version tag.
>
> A possible way to handle this in a way that is not dependent on Git
> would be to store the originating n-v-r in the abixml directly.  The
> tool that emits the abixml (from the original package) would be able to
> do that.
>
>> --------------------
>>
>> For using this data, I envision abidiff / abicompat taking a new form
>> for its right operand.  It could be a git url identifying the distro
>> branch or tag.  libabigail would fetch the corresponding file.xml
>> within that.  Simplify/default the heck out of it for ease of use:
>>
>>    export $BRANCH=fedora/39/x86_64
>>    abicompat /bin/myprogram gitabixml:$BRANCH
>>
>> (Where "gitabixml:" could instruct the tool to look at the sourceware
>> libabigail git / gitweb / cgit server.  Let users specify different or
>> private git servers via environment variables or something.
> Yes, something like that.  I guess the specific will depend on what we
> end up settling on for the above (and below).
>
>
>> --------------------
>>
>> For collecting this data, I envision writing some distro-specific
>> scripts, kind of like fedabipkgdiff, being run by contributors or
>> ourselves.  One flavour could run in operational installed distros,
>> doing the equivalent of
>>
>>      find $PATHS -name '*.so.*' | while read lib; do
>>         # or filter with elfclassify
>>         package=`rpm -qf "$lib"`
>>         abidw "$lib" | (cd $gitrepo/`dirname $lib`; cat > "$lib.xml")
>>         (cd $gitrepo; git commit -m"Package: $package" "$lib.xml")
>>      done
>>
>> and rerun that occasionally as updates flow down from the distro.
>> This could be done on a single beefy box running containers with
>> different distros.
> Yes, I like the idea of having something incremental like this.
>
> There would probably be some tweaks added to abidw to let it add a
> "version string" (the N-V-R mentioned earlier) to the abidw, but these
> are details.  Also, we need to add a mode to libabigail to let it expect
> debuginfod to find the debuginfo because today, it expects the user to
> provide the debug info location in cases where it's not already
> installed on the system.  Again, this is a detail but it's going to
> matter as soon as we the rubber hits the road.
>
>> Another flavour could be to take a set of RPM/etc. archives on a
>> filesystem (or an ISO image), incrementally decompress them, run abidw
>> on the individual files, and similarly construct the git repo of
>> abixml files.  (This is kind of like how debuginfod produces indexes
>> from a bunch of RPMs.)
> ACK.
>> No matter how the local git repo is populated, each branch describing
>> a data contributor's distro could be pushed to the central one,
>> bringing that one up to date.  Patches representing updates could be
>> emailed too, but no one will want to read/review that stuff.  We'd
>> probably need a trusted pool of contributors who can just commit to
>> areas of the central git repo.  Secured with gitsigur of course. :-)
>>
>> The central repo could be built up entirely gradually.  If some
>> libraries were omitted from initial commits for a distro, a later
>> contribution could fill in the gaps.
> ACK.
>
> Thank you for putting this thoughts together.  The idea is getting much
> less abstract in my mind now.
>
> Cheers,
>

Re: idea: abigail abixml archive

[Dodji Seketeli]

Ben Woodard <woodard@redhat.com> a écrit:

>> If I understand things correctly, you would need to:
>>
>>      1/ Get the ABI of the (transitive closure) set of dependencies of
>>      the binary on its original distro.  I call it orig-deps-abi.
>>
>>      2/ Get the ABI of the set of dependencies of the binary on the
>>      target distro.  I call it target-deps-abi.
>>
>>      3/ compare target-deps-abi against orig-deps-abi.
>>
>> abicompat, for instance, hasn't been designed for that.  We would need
>> to either extend it to support 1-3 or come up with a new tool for that.
>
> I continue to believe that the current design of abicompat is not
> sufficient for what users want to do. The name is misleading and what
> it currently accomplishes not only doesn't match user's expectations
> given its name, What it does accomplish is not very useful from a
> user's perspective. We have literally not been able to come up with an
> actual use for abicompat as it now exists without putting a ton of
> scripting around it. Therefore I strongly believe that it should be
> extended to actually compute the forward and backward ABI dependencies
> of the program and its library dependencies. This is what users 
> expect abicompat to do before they are educated on what it actually
> does and this is what users need it to do.

FWIW, as not everyone is familiar with the requests for enhancement
about abicompat, Ben has filed a number of these:

    * https://sourceware.org/bugzilla/show_bug.cgi?id=27514
      "recursively compare libraries in abicompat"
      (This is somehow asking for 1/ above)

    * https://sourceware.org/bugzilla/show_bug.cgi?id=27208
      "abicompat doesn't consider references from the libraries to the main executable"
       This one depends on:
        * https://sourceware.org/bugzilla/show_bug.cgi?id=29008
          "abicompat doesn't consider functions (or variables) whose symbol type is NOTYPE"

    * https://sourceware.org/bugzilla/show_bug.cgi?id=29013
      "abicompat doesn't consider weak symbol replacement"

[...]

Cheers,

-- 
		Dodji

Re: idea: abigail abixml archive

[Frank Ch. Eigler]

Hi -

Having gone quiet for a few weeks, would like to show a really early
sketch of this gadget.  See the tools/abidb python script in the
users/fche/abidb branch.  I am better educated about limitations of
abicompat as discussed downthread, but this script doesn't attempt to
solve the whole problem yet.  Recursive searches for SOLIB chains,
reverse dependencies for weak symbols etc., are not in there, but
some of them could be added.  Anyway, what there is now:


% abidb --help
usage: abidb [-h] [--loglevel LOGLEVEL] [--git GIT]
             [--distrobranch DISTROBRANCH] [--submit [SUBMIT ...]]
             [--check [CHECK ...]] [--ld-library-path LD_LIBRARY_PATH]

Check binary against abidb corpus and/or submit new data.

options:
  -h, --help            show this help message and exit
  --loglevel LOGLEVEL   logging level (default: info)
  --git GIT             abidb git working tree (default: .)
  --distrobranch DISTROBRANCH
                        use given abidb distrobranch (default:
                        fedora/39/x86_64)
  --submit [SUBMIT ...]
                        submit abidw of given binaries to abidb (default: [])
  --check [CHECK ...]   check given binaries against abidb (default: [])
  --ld-library-path LD_LIBRARY_PATH
                        LD_LIBRARY_PATH to use for soname resolution during
                        check (default: /lib:/lib64)


% git init /tmp/abidb

NB: this git repo would naturally be centrally shareable over time.


% abidb --git /tmp/abidb --submit /lib*/libc.so.?
2023-12-05 19:07:58,738:abidb:INFO:checked out distrobranch fedora/39/x86_64 files 0
2023-12-05 19:07:59,268:abidb:INFO:binary /lib64/libc.so.6 abidb-path /tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml abixml-length 3305422 commit e90283c5fe45a2c465edfeb2ec55f3b47cf6dce3
2023-12-05 19:08:00,226:abidb:INFO:binary /lib/libc.so.6 abidb-path /tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml abixml-length 3521035 commit 57536c5b38e9463654432c0b31e4ff5401eed512

NB: the directory structure in git mirrors the filesystem, plus the
appropriate BUILDID.  This allows multiple different updates of the
same library to be represented.

% abidb  --git /tmp/abidb --check /bin/ls         
2023-12-05 19:08:35,699:abidb:INFO:checked out distrobranch fedora/39/x86_64 files 2
2023-12-05 19:08:35,701:abidb:WARNING:no abi.xml found for libselinux.so.1 searching under /lib:/lib64
2023-12-05 19:08:35,701:abidb:WARNING:no abi.xml found for libcap.so.2 searching under /lib:/lib64
2023-12-05 19:08:35,867:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml
2023-12-05 19:08:36,022:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml

NB: it complains not having a few solibs in the database yet, so let's add them:


% abidb  --git /tmp/abidb --submit /lib*/libselinux.so.? /lib*/libcap.so.?
2023-12-05 19:09:10,984:abidb:INFO:checked out distrobranch fedora/39/x86_64 files 2
2023-12-05 19:09:11,041:abidb:INFO:binary /lib64/libselinux.so.1 abidb-path /tmp/abidb/lib64/libselinux.so.1/c3ca2b00f1505a173d44417dc98dced2ec390268.xml abixml-length 281444 commit 91e92229bf17b00cb708d5f0da3e9d94708665a2
2023-12-05 19:09:11,087:abidb:INFO:binary /lib/libselinux.so.1 abidb-path /tmp/abidb/lib/libselinux.so.1/b4ea9035186f94416600ccf6b378afb408a8ec8e.xml abixml-length 280693 commit 9979a994ef5d77ab25ce777ab22ff10aae3d4f59
2023-12-05 19:09:11,119:abidb:INFO:binary /lib64/libcap.so.2 abidb-path /tmp/abidb/lib64/libcap.so.2/ff26c67e54bb86d066f1c3cee3cef74fe0c13a77.xml abixml-length 68942 commit 780a099d2a59132aca0f82ab6d5e553be08c2969
2023-12-05 19:09:11,144:abidb:INFO:binary /lib/libcap.so.2 abidb-path /tmp/abidb/lib/libcap.so.2/86ab6783bd93f43139db2924cbbf25bc9419dc9c.xml abixml-length 69053 commit 7f9cec82580c58007748f470408030195d9dcd07

NB: now it will be happy

% abidb  --git /tmp/abidb --check /bin/ls                                 
2023-12-05 19:09:27,561:abidb:INFO:checked out distrobranch fedora/39/x86_64 files 6
2023-12-05 19:09:27,578:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib/libselinux.so.1/b4ea9035186f94416600ccf6b378afb408a8ec8e.xml
2023-12-05 19:09:27,590:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib64/libselinux.so.1/c3ca2b00f1505a173d44417dc98dced2ec390268.xml
2023-12-05 19:09:27,596:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib/libcap.so.2/86ab6783bd93f43139db2924cbbf25bc9419dc9c.xml
2023-12-05 19:09:27,602:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib64/libcap.so.2/ff26c67e54bb86d066f1c3cee3cef74fe0c13a77.xml
2023-12-05 19:09:27,768:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml
2023-12-05 19:09:27,920:abidb:INFO:abicompat success: /bin/ls vs. /tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml

NB: ok let's try some random ancient binary

% abidb  --git /tmp/abidb --check /usr/local/bin/avplay

2023-12-05 19:11:17,151:abidb:INFO:checked out distrobranch fedora/39/x86_64 files 6
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libm.so.6 searching under /lib:/lib64
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libasound.so.2 searching under /lib:/lib64
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libbz2.so.1 searching under /lib:/lib64
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libz.so.1 searching under /lib:/lib64
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libSDL-1.2.so.0 searching under /lib:/lib64
2023-12-05 19:11:17,153:abidb:WARNING:no abi.xml found for libpthread.so.0 searching under /lib:/lib64
2023-12-05 19:11:22,889:abidb:WARNING:abicompat error: /usr/local/bin/avplay vs. /tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml:
functions defined in library '/tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml'
have sub-types that are different from what application '/usr/local/bin/avplay' expects:

  function int __xpg_strerror_r(int, char*, size_t):
    parameter 3 of type 'typedef size_t' changed:
      underlying type 'unsigned long int' changed:
        type name changed from 'unsigned long int' to 'unsigned int'
        type size changed from 64 to 32 (in bits)
    address size of function changed from 64 bits to 32 bits

variables defined in library '/tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml'
have sub-types that are different from what application '/usr/local/bin/avplay' expects:

  void (void*, void*)* __free_hook:
    in pointed to type 'function type void (void*, void*)':
      address size of function changed from 64 bits to 32 bits

  FILE* stdin:
    in pointed to type 'typedef FILE' at FILE.h:7:1:
      underlying type 'struct _IO_FILE' at struct_FILE.h:49:1 changed:
        type size changed from 1728 to 1184 (in bits)
[...]
  long int __timezone:
    type size changed from 64 to 32 (in bits)



2023-12-05 19:11:22,889:abidb:INFO:abicompat success: /usr/local/bin/avplay vs. /tmp/abidb/lib/libc.so.6/02ac2b73b0b28086109ca73389bf97a26fd70542.xml
2023-12-05 19:11:28,727:abidb:WARNING:abicompat error: /usr/local/bin/avplay vs. /tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml:
functions defined in library '/tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml'
have sub-types that are different from what application '/usr/local/bin/avplay' expects:

  function int fputc(int, FILE*):
    parameter 2 of type 'FILE*' has sub-type changes:
      in pointed to type 'typedef FILE' at FILE.h:7:1:
        underlying type 'struct _IO_FILE' at struct_FILE.h:49:1 changed:
          type size hasn't changed
[...]
        unqualified underlying type 'struct tm' changed at struct_tm.h:7:1, as reported earlier

variables defined in library '/tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml'
have sub-types that are different from what application '/usr/local/bin/avplay' expects:

  FILE* stdin:
    pointed to type 'typedef FILE' changed at FILE.h:7:1, as reported earlier
[...]

2023-12-05 19:11:28,727:abidb:INFO:abicompat success: /usr/local/bin/avplay vs. /tmp/abidb/lib64/libc.so.6/7dd93cb9991a89f0ec53d9443a0b78ad952269bc.xml


That's it for now.


- FChE