Re: [PATCH v5 1/2] check-uapi: Introduce check-uapi.sh

[Masahiro Yamada <masahiroy@kernel.org>]

On Sat, Apr 8, 2023 at 5:35 AM John Moon <quic_johmoo@quicinc.com> wrote:
>
> While the kernel community has been good at maintaining backwards
> compatibility with kernel UAPIs, it would be helpful to have a tool
> to check if a commit introduces changes that break backwards
> compatibility.
>
> To that end, introduce check-uapi.sh: a simple shell script that
> checks for changes to UAPI headers using libabigail.
>
> libabigail is "a framework which aims at helping developers and
> software distributors to spot some ABI-related issues like interface
> incompatibility in ELF shared libraries by performing a static
> analysis of the ELF binaries at hand."
>
> The script uses one of libabigail's tools, "abidiff", to compile the
> changed header before and after the commit to detect any changes.
>
> abidiff "compares the ABI of two shared libraries in ELF format. It
> emits a meaningful report describing the differences between the two
> ABIs."
>
> The script also includes the ability to check the compatibility of
> all UAPI headers across commits. This allows developers to inspect
> the stability of the UAPIs over time.
>
> Signed-off-by: John Moon <quic_johmoo@quicinc.com>



BTW, is there anybody (except the submitters) who loves this tool?
(or anybody who has ever evaluated this?)

I am the only person who pointed out something.



I put some more comments from users' point of view.





[1] The number of lines of the error log.


According to this tool, it looks like we broke a lot of UAPI
headers in the previous MW (between v6.2 and v6.3-rc1).

Not only in the previous MW, we always broke so many UAPI
headers, this script says.


The following command outputs 2380 lines.


$ ./scripts/check-uapi.sh -q -b v6.3-rc1 -p v6.2 2>&1  | wc
   2380    9438  104953


I do not know how many are real breakages, and
how many are false positives.

Anyway, I will not check the 2380 lines of the error log.


And, please note that the help message explains '-q' is the *quiet* mode.
It is the opposite. This is super noisy.




[2] Be careful!


While testing the patch submissions of this,
I messed up my repository multiple times.

The script takes some time because it builds many objects
internally.

However, once this script starts running, you must not hit Ctrl-C.
If you do it, your repository will be sprinkled with a ton
of untracked files.

Apply this patch, and run "./scripts/check-uapi.sh -p v6.0"
and hit Ctrl-C.

Repeat it a couple of times, and "git status" will show you
something horrible.


You will never know when git is checking out a commit
because this script hides it by 'git checkout --quiet'.


So, this tool should show a caveat at least.


'git checkout' should not be hidden, or
maybe a caveat message should be shown.


CAVEAT
This tool runs 'git checkout' a couple of times internally.
If you interrupt it, your worktree might be messed up.












--
Best Regards
Masahiro Yamada