[Bug default/20514] New: Write a kabidiff utility to compare the ABI of Linux Kernel trees

[Bug default/20514] New: Write a kabidiff utility to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

            Bug ID: 20514
           Summary: Write a kabidiff utility to compare the ABI of Linux
                    Kernel trees
           Product: libabigail
           Version: unspecified
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: default
          Assignee: dodji at redhat dot com
          Reporter: dodji at redhat dot com
                CC: libabigail at sourceware dot org
  Target Milestone: ---

It would be useful to have an ABI comparison tool, just like abidiff, that
would compare the ABI exposed by the Linux kernel to its modules.

As an initial step, I guess we can consider functions (and global variables)
that are considered part of the kernel's internal ABI to be those marked by the
macros EXPORT_SYMBOL and EXPORT_SYMBOL_GPL.

The tool would consider a kernel "tree" to be the union of the kernel binary
and the accompanying modules that are intended to be loaded by it.

As a first step, though, just considering the kernel binary would be a good
proof of concept of what we can come up with.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a kabidiff utility to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

dodji at redhat dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |andrei.moscow at mail dot ru

--- Comment #2 from dodji at redhat dot com ---
*** Bug 20681 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a kabidiff utility to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

dodji at redhat dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #1 from dodji at redhat dot com ---
Generally speaking, work has started to make libabigail understand linux kernel
(and module) binaries.

This is happening in the branch dodji/kabidiff of the git repostory.  You can
browse it at
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=shortlog;h=refs/heads/dodji/kabidiff.

For now, the code in that branch has the following changes:

* support of a new flag --linux-kernel-mode for the abidiff binary.

  This new flag lets abidiff load two Linux kernel binaries and compare them.

* support of a new flag --linux-kernel-mode for the abidw binary.

  This new flag lets abidw load a Linux kernel binary and emit a textual
representation of its ABI on standard output.

* support of a new flag --linux-kernel-abi-whitelist <path-to-whitelist-file>

  This new flag lets abidiff load an abi-whitelist file that lists the names of
the functions that represent the abi of the kernel (and its modules) that
abidiff should take into account.  Any other function of the kernel is dropped
on the floor and not considered when comparing binaries.

The code also sports many changes including time and space optimization to make
libabigail load ABI representation of binaries, compare them and report their
changes in a timely manner and using a reasonable amount of memory.

These changes are building blocks that are going to be used in to write a tool
that is able to compare the ABI of a set made of a kernel and its modules
against another set of kernel + modules.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

dodji at redhat dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Write a kabidiff utility to |Write a command line tool
                   |compare the ABI of Linux    |to compare the ABI of Linux
                   |Kernel trees                |Kernel trees

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

--- Comment #3 from dodji at redhat dot com ---
The perimeter of this task is changing and is getting more clear.

So now, here is what I am planning/doing.

1/ make the normal abidiff understand Linux Kernel binaries -- either vmlinux
or Kernel modules.  This is almost done now.  I just need to cleanup some
option handling of abidiff, write some documentation and perform some testing.

2/ make abipkgdiff to detect that the packages it is looking at are linux
kernel packages.  On Fedora/el6 at least, if it's looking at linux kernel
packages along with their debug info packages, the vmlinux binary is present in
the debug info package, un-compressed, with all its debug info non-stripped and
non-split.  So it should look in the debug info package to get the two vmlinux
binaries to compare.  Kernel *Modules* will have to be handled in the usual
way.  Except that they are usually compressed, so the tool will need to
un-compress them before comparing them.  I haven't started this yet.

3/ write a new tool, probably called kmidiff that will take two kernel *trees*.
 The ABI of a kernel tree is going to be represented as the union of the ABIs
of the vmlinux binary and the kernel module binaries.  One such union is going
to be called a Composite ABI Corpus.  kmidiff is thus going to compare
Composite ABI corpora.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

--- Comment #4 from dodji at redhat dot com ---
(In reply to dodji from comment #3)
> The perimeter of this task is changing and is getting more clear.
> 
> 1/ make the normal abidiff understand Linux Kernel binaries -- either
> vmlinux or Kernel modules.  This is almost done now.  I just need to cleanup
> some option handling of abidiff, write some documentation and perform some
> testing.

abidiff and abidw support handling Linux Kernel binaries by default.  No need
to provide any command line option.  Just give these tools a decompressed
vmlinux binary compiled with debug info and voila.

This has been merged into master.  The relevant commits are
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commit;h=fadfa1a6f6a18b46e7a4164bd87743bd5202054e
and
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commit;h=08e760221bf3473761d1ed330be09c0b7cb01a1c.

This is the first step of the plan we outlined to support Linux Kernel binaries
and Linux Kernel development work flows in general.

There still are tasks to complete for this enhancement request to be done.
Details are available in comment
https://sourceware.org/bugzilla/show_bug.cgi?id=20514#c3.

Those tasks are going to be worked on on in the dodji/kabidiff branch as in the
past.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

--- Comment #5 from dodji at redhat dot com ---
In the dodji/kabidiff branch, there is now the kmidiff tool that can compare
two linux kernel trees.  A kernel tree is a source tree in which the user has
built the linux kernel.

The documentation for that tool is in the file doc/manuals/kmidiff.rst, on that
branch.  When that branch is merged into master and 1.0.rc7 is released, that
documentation should appear on
https://sourceware.org/libabigail/manual/libabigail-tools.html#tools-manuals

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

dodji at redhat dot com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #6 from dodji at redhat dot com ---
The dodji/kabidiff branch has been merged into master, thus closing this
tracking bug.

As a result, master now has a new kmidiff tool that compares the "Kernel/Module
Interface" of two Linux kernel tree build.  A Linux Kernel Tree build is the
result of cloning *and* compiling a Linux kernel source tree.

During the comparison the tool considers the union of the vmlinux binary and
all the kernel module as being just one Kernel.  And it's that Kernel that is
compared to it's peer Kernel from the second tree.

What remains to be done now is to make abipkgdiff detect Linux kernel packages
specifically and perform similar comparisons on them.  But this is going to be
tracked by a specific bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[Bug default/20514] Write a command line tool to compare the ABI of Linux Kernel trees

[dodji at redhat dot com]

https://sourceware.org/bugzilla/show_bug.cgi?id=20514

--- Comment #7 from dodji at redhat dot com ---
The bug to track the modification to make abipkgdiff compare Linux Kernel
packages as expected is #21730

-- 
You are receiving this mail because:
You are on the CC list for the bug.