Re: Getting a non-zero exit code from abi-diff when breakage is detected.

Re: Getting a non-zero exit code from abi-diff when breakage is detected.

[Dodji Seketeli]

Hello Eric,

Eric Fiselier <eric@efcs.ca> a écrit:

> I would like to request a feature (assuming it doesn't already exist).
> I would like to be able to make abidiff return a non-zero exit code if
> it detects breaking changes.

This makes sense.

> Currently I have a buildbot setup that runs abidiff for every revision
> of libc++.

Whoah!

> The goal is to have this buildbot detect the results of the abidiff
> step and properly report the failure. The easiest way to do this on my
> end is for abidiff to return non-zero.

I agree.

> Does this sound like a reasonable feature?

Of course it does :-)

So please, lemme me put this in perspective so that we can come up with
a modus operandi going forward.

Today, the abidiff report is made of roughly three parts:

    1/ a part that reports exported and defined functions/variables that
    got removed.

    2/ a part that reports exported and defined functions/variables that
    got added.

    3/ a part that reports exported and defined functions/variables that
    have sub-type changes.

In an ABI compatibility checking mode, the abidiff program could return
a non-zero value if part 1/ is non-empty.  The hard part would be to
deal with part 3/; by definition, it's a grey-area; there are changes
exhibited in that part that need human interpretation to determine if
they are ABI incompatible or not.  But I believe that libabigail has
most of the infrastructure needed to write heuristics that will
progressively help bring us to a point where a change showing up in part
3/ is highly likely to be an ABI-incompatible change.  Also, we already
have support for suppression specifications so that users can make
abidiff recognize (and filter out) patterns of ABI-compatible changes.

So I think we can start with, say, a --check-abi-compat (or something
like that) command line option that would make abidiff return a non-zero
value if either parts 1/ or 3/ are non-empty.  And we'd gradually add
more oumph to the filtering heuristics used in part 3/.

What do you think?

Cheers,

-- 
		Dodji

Re: Getting a non-zero exit code from abi-diff when breakage is detected.

[Dodji Seketeli]

Eric Fiselier <eric@efcs.ca> a écrit:

>>So I think we can start with, say, a --check-abi-compat (or something
>>like that) command line option that would make abidiff return a non-zero
>>value if either parts 1/ or 3/ are non-empty.  And we'd gradually add
>>more oumph to the filtering heuristics used in part 3/.
>
> That sounds great to me and would fit my use case perfectly.

Okay, cool.

I have filed an enhancement request for this task so that we can track
it: https://sourceware.org/bugzilla/show_bug.cgi?id=18146.

Cheers,

-- 
		Dodji

Getting a non-zero exit code from abi-diff when breakage is detected.

[Eric Fiselier]

Hi all,

I would like to request a feature (assuming it doesn't already exist).
I would like to be able to make abidiff return a non-zero exit code if
it detects breaking changes.

Currently I have a buildbot setup that runs abidiff for every revision
of libc++. The goal is to have this buildbot detect the results of the
abidiff step and properly report the failure. The easiest way to do
this on my end is for abidiff to return non-zero.

Does this sound like a reasonable feature?

/Eric

Re: Getting a non-zero exit code from abi-diff when breakage is detected.

[Eric Fiselier]

>So I think we can start with, say, a --check-abi-compat (or something
>like that) command line option that would make abidiff return a non-zero
>value if either parts 1/ or 3/ are non-empty.  And we'd gradually add
>more oumph to the filtering heuristics used in part 3/.

That sounds great to me and would fit my use case perfectly.

/Eric

On Thu, Mar 19, 2015 at 6:53 AM, Dodji Seketeli <dodji@seketeli.org> wrote:
> Hello Eric,
>
> Eric Fiselier <eric@efcs.ca> a écrit:
>
>> I would like to request a feature (assuming it doesn't already exist).
>> I would like to be able to make abidiff return a non-zero exit code if
>> it detects breaking changes.
>
> This makes sense.
>
>> Currently I have a buildbot setup that runs abidiff for every revision
>> of libc++.
>
> Whoah!
>
>> The goal is to have this buildbot detect the results of the abidiff
>> step and properly report the failure. The easiest way to do this on my
>> end is for abidiff to return non-zero.
>
> I agree.
>
>> Does this sound like a reasonable feature?
>
> Of course it does :-)
>
> So please, lemme me put this in perspective so that we can come up with
> a modus operandi going forward.
>
> Today, the abidiff report is made of roughly three parts:
>
>     1/ a part that reports exported and defined functions/variables that
>     got removed.
>
>     2/ a part that reports exported and defined functions/variables that
>     got added.
>
>     3/ a part that reports exported and defined functions/variables that
>     have sub-type changes.
>
> In an ABI compatibility checking mode, the abidiff program could return
> a non-zero value if part 1/ is non-empty.  The hard part would be to
> deal with part 3/; by definition, it's a grey-area; there are changes
> exhibited in that part that need human interpretation to determine if
> they are ABI incompatible or not.  But I believe that libabigail has
> most of the infrastructure needed to write heuristics that will
> progressively help bring us to a point where a change showing up in part
> 3/ is highly likely to be an ABI-incompatible change.  Also, we already
> have support for suppression specifications so that users can make
> abidiff recognize (and filter out) patterns of ABI-compatible changes.
>
> So I think we can start with, say, a --check-abi-compat (or something
> like that) command line option that would make abidiff return a non-zero
> value if either parts 1/ or 3/ are non-empty.  And we'd gradually add
> more oumph to the filtering heuristics used in part 3/.
>
> What do you think?
>
> Cheers,
>
> --
>                 Dodji

Re: Getting a non-zero exit code from abi-diff when breakage is detected.

[Dodji Seketeli]

Hello Eric,

Eric Fiselier <eric@efcs.ca> a écrit:

> I would like to request a feature (assuming it doesn't already exist).
> I would like to be able to make abidiff return a non-zero exit code if
> it detects breaking changes.

[...]

Dodji Seketeli <dodji@seketeli.org> a écrit:

> I have filed an enhancement request for this task so that we can track
> it: https://sourceware.org/bugzilla/show_bug.cgi?id=18146.

I have implemented this and pushed it to the master branch of our source
repository.  Hopefully it looks like what you had in mind.  Otherwise, I
am all ears :-)

Thank you for raising this, Eric.

Cheers,

-- 
		Dodji