* Re: a new tool fedabipkgdiff
2016-01-01 0:00 a new tool fedabipkgdiff Chenxiong Qi
@ 2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Chenxiong Qi
1 sibling, 1 reply; 10+ messages in thread
From: Chenxiong Qi @ 2016-01-01 0:00 UTC (permalink / raw)
To: libabigail, dodji
[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]
Hi,
This is an update version for the new tool fedabipkgdiff. Thanks for Mr.
Dodji's help and patch to enable testing and installation of
fedabipkgdiff. Lots of fixes are done, and now --all-subpackages option
is supported in the 4th use case mentioned, that is all non-noarch RPM
packages within a build could be checked with abipkgdiff. Please review
and any feedback is appreciated.
On 02/17/2016 06:38 PM, Chenxiong Qi wrote:
> Hi,
>
> This is a new tool fedabipkgdiff that would be much convenient for
> Fedora packagers to check potential ABI/API differences quickly using
> abipkgdiff shipped with libabigail. This tool came from a cool idea
> from Dodji. Currently, as the first step, it supports following ways,
>
> fedabipkgdiff --from fc23 ./foo-0.1-1.fc23.x86_64.rpm
> fedabipkgdiff --from fc23 --to fc24 foo
> fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
> fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
>
> For more details, please refer to
> https://sourceware.org/bugzilla/show_bug.cgi?id=19428
>
> Next step is to support the 4th use case mentioned in bug 19428.
>
> fedabipkgdiff is being under development, still need to improve.
> Welcome any feedback. Thanks.
>
> Happy hacking :)
>
> Regards,
> Chenxiong Qi
--
Regards,
Chenxiong Qi
[-- Attachment #2: 0001-new-tool-of-fedabipkgdiff.patch --]
[-- Type: text/x-patch, Size: 64597 bytes --]
From 42761dc622218060cd6d7dfdf8774d6189c8788b Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Tue, 9 Feb 2016 18:05:33 +0800
Subject: [PATCH] new tool of fedabipkgdiff
fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
compatibility issues quickly.
Currently with the first version of fedabipkgdiff, you can invoke it in
following ways.
fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
fedabipkgdiff --all-subpackages foo-0.1-1.fc23 foo-0.1-1.fc24
* autoconf-archive/ax_compare_version.m4: New file copied from the
autoconf-archive project.
* autoconf-archive/ax_prog_python_version.m4: Likewise.
* autoconf-archive/ax_python_module.m4: Likewise.
* Makefile.am: Add the new files above to the source distribution.
* configure.ac: Include the new m4 macros from the autoconf
archive. Add a new --enable-fedabipkgdiff option. Update the
report at the end of the configure process to show the status of
the fedabipkgdiff feature. Add check for prerequisite python modules
itertools, shutil, unittest and mock. These are necessary for the
unit test of fedabipkgdiff. Generate tests/runtestfedabipkgdiff.py into the
build directory, from the tests/runtestfedabipkgdiff.py.in input file.
* tools/Makefile.am: Include the fedabipkgdiff to the source
distribution and install it if the "fedabipkgdiff" feature is
enabled.
* tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
autoconf template file in here.
* tests/runtestfedabipkgdiff.py.in: New unit test file.
* tools/fedabipkgdiff: New tool fedabipkgdiff.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
---
.gitignore | 6 +
Makefile.am | 3 +
autoconf-archive/ax_compare_version.m4 | 177 +++++++
autoconf-archive/ax_prog_python_version.m4 | 66 +++
autoconf-archive/ax_python_module.m4 | 56 ++
configure.ac | 86 ++-
tests/Makefile.am | 6 +-
tests/runtestfedabipkgdiff.py.in | 450 ++++++++++++++++
tools/Makefile.am | 6 +
tools/fedabipkgdiff | 805 +++++++++++++++++++++++++++++
10 files changed, 1659 insertions(+), 2 deletions(-)
create mode 100644 autoconf-archive/ax_compare_version.m4
create mode 100644 autoconf-archive/ax_prog_python_version.m4
create mode 100644 autoconf-archive/ax_python_module.m4
create mode 100755 tests/runtestfedabipkgdiff.py.in
create mode 100755 tools/fedabipkgdiff
diff --git a/.gitignore b/.gitignore
index bb7c42a..a60cadb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ Makefile.in
*.lo
*.o
*~
+*.swp
/aclocal.m4
/autom4te.cache/
@@ -17,3 +18,8 @@ Makefile.in
/include/abg-version.h
/*.pc
+
+.tags
+build/
+TAGS
+fedabipkgdiffc
\ No newline at end of file
diff --git a/Makefile.am b/Makefile.am
index c855cf6..1ae2290 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -12,6 +12,9 @@ pkgconfig_DATA = libabigail.pc
#dist_bashcompletion_DATA =
EXTRA_DIST = \
+autoconf-archive/ax_python_module.m4 \
+autoconf-archive/ax_prog_python_version.m4 \
+autoconf-archive/ax_compare_version.m4 \
NEWS README COPYING ChangeLog \
COPYING-LGPLV2 COPYING-LGPLV3 \
COPYING-GPLV3 gen-changelog.py \
diff --git a/autoconf-archive/ax_compare_version.m4 b/autoconf-archive/ax_compare_version.m4
new file mode 100644
index 0000000..74dc0fd
--- /dev/null
+++ b/autoconf-archive/ax_compare_version.m4
@@ -0,0 +1,177 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_compare_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_COMPARE_VERSION(VERSION_A, OP, VERSION_B, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# This macro compares two version strings. Due to the various number of
+# minor-version numbers that can exist, and the fact that string
+# comparisons are not compatible with numeric comparisons, this is not
+# necessarily trivial to do in a autoconf script. This macro makes doing
+# these comparisons easy.
+#
+# The six basic comparisons are available, as well as checking equality
+# limited to a certain number of minor-version levels.
+#
+# The operator OP determines what type of comparison to do, and can be one
+# of:
+#
+# eq - equal (test A == B)
+# ne - not equal (test A != B)
+# le - less than or equal (test A <= B)
+# ge - greater than or equal (test A >= B)
+# lt - less than (test A < B)
+# gt - greater than (test A > B)
+#
+# Additionally, the eq and ne operator can have a number after it to limit
+# the test to that number of minor versions.
+#
+# eq0 - equal up to the length of the shorter version
+# ne0 - not equal up to the length of the shorter version
+# eqN - equal up to N sub-version levels
+# neN - not equal up to N sub-version levels
+#
+# When the condition is true, shell commands ACTION-IF-TRUE are run,
+# otherwise shell commands ACTION-IF-FALSE are run. The environment
+# variable 'ax_compare_version' is always set to either 'true' or 'false'
+# as well.
+#
+# Examples:
+#
+# AX_COMPARE_VERSION([3.15.7],[lt],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[lt],[3.15.8])
+#
+# would both be true.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[gt],[3.15.8])
+#
+# would both be false.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq2],[3.15.8])
+#
+# would be true because it is only comparing two minor versions.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq0],[3.15])
+#
+# would be true because it is only comparing the lesser number of minor
+# versions of the two values.
+#
+# Note: The characters that separate the version numbers do not matter. An
+# empty string is the same as version 0. OP is evaluated by autoconf, not
+# configure, so must be a string, not a variable.
+#
+# The author would like to acknowledge Guido Draheim whose advice about
+# the m4_case and m4_ifvaln functions make this macro only include the
+# portions necessary to perform the specific comparison specified by the
+# OP argument in the final configure script.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Tim Toolan <toolan@ele.uri.edu>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+dnl #########################################################################
+AC_DEFUN([AX_COMPARE_VERSION], [
+ AC_REQUIRE([AC_PROG_AWK])
+
+ # Used to indicate true or false condition
+ ax_compare_version=false
+
+ # Convert the two version strings to be compared into a format that
+ # allows a simple string comparison. The end result is that a version
+ # string of the form 1.12.5-r617 will be converted to the form
+ # 0001001200050617. In other words, each number is zero padded to four
+ # digits, and non digits are removed.
+ AS_VAR_PUSHDEF([A],[ax_compare_version_A])
+ A=`echo "$1" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ AS_VAR_PUSHDEF([B],[ax_compare_version_B])
+ B=`echo "$3" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ dnl # In the case of le, ge, lt, and gt, the strings are sorted as necessary
+ dnl # then the first line is used to determine if the condition is true.
+ dnl # The sed right after the echo is to remove any indented white space.
+ m4_case(m4_tolower($2),
+ [lt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [gt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [le],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],
+ [ge],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],[
+ dnl Split the operator from the subversion count if present.
+ m4_bmatch(m4_substr($2,2),
+ [0],[
+ # A count of zero means use the length of the shorter version.
+ # Determine the number of characters in A and B.
+ ax_compare_version_len_A=`echo "$A" | $AWK '{print(length)}'`
+ ax_compare_version_len_B=`echo "$B" | $AWK '{print(length)}'`
+
+ # Set A to no more than B's length and B to no more than A's length.
+ A=`echo "$A" | sed "s/\(.\{$ax_compare_version_len_B\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(.\{$ax_compare_version_len_A\}\).*/\1/"`
+ ],
+ [[0-9]+],[
+ # A count greater than zero means use only that many subversions
+ A=`echo "$A" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ ],
+ [.+],[
+ AC_WARNING(
+ [illegal OP numeric parameter: $2])
+ ],[])
+
+ # Pad zeros at end of numbers to make same length.
+ ax_compare_version_tmp_A="$A`echo $B | sed 's/./0/g'`"
+ B="$B`echo $A | sed 's/./0/g'`"
+ A="$ax_compare_version_tmp_A"
+
+ # Check for equality or inequality as necessary.
+ m4_case(m4_tolower(m4_substr($2,0,2)),
+ [eq],[
+ test "x$A" = "x$B" && ax_compare_version=true
+ ],
+ [ne],[
+ test "x$A" != "x$B" && ax_compare_version=true
+ ],[
+ AC_WARNING([illegal OP parameter: $2])
+ ])
+ ])
+
+ AS_VAR_POPDEF([A])dnl
+ AS_VAR_POPDEF([B])dnl
+
+ dnl # Execute ACTION-IF-TRUE / ACTION-IF-FALSE.
+ if test "$ax_compare_version" = "true" ; then
+ m4_ifvaln([$4],[$4],[:])dnl
+ m4_ifvaln([$5],[else $5])dnl
+ fi
+]) dnl AX_COMPARE_VERSION
diff --git a/autoconf-archive/ax_prog_python_version.m4 b/autoconf-archive/ax_prog_python_version.m4
new file mode 100644
index 0000000..628a3e4
--- /dev/null
+++ b/autoconf-archive/ax_prog_python_version.m4
@@ -0,0 +1,66 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_prog_python_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PROG_PYTHON_VERSION([VERSION],[ACTION-IF-TRUE],[ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# Makes sure that python supports the version indicated. If true the shell
+# commands in ACTION-IF-TRUE are executed. If not the shell commands in
+# ACTION-IF-FALSE are run. Note if $PYTHON is not set (for example by
+# running AC_CHECK_PROG or AC_PATH_PROG) the macro will fail.
+#
+# Example:
+#
+# AC_PATH_PROG([PYTHON],[python])
+# AX_PROG_PYTHON_VERSION([2.4.4],[ ... ],[ ... ])
+#
+# This will check to make sure that the python you have supports at least
+# version 2.4.4.
+#
+# NOTE: This macro uses the $PYTHON variable to perform the check.
+# AX_WITH_PYTHON can be used to set that variable prior to running this
+# macro. The $PYTHON_VERSION variable will be valorized with the detected
+# version.
+#
+# LICENSE
+#
+# Copyright (c) 2009 Francesco Salvestrini <salvestrini@users.sourceforge.net>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+AC_DEFUN([AX_PROG_PYTHON_VERSION],[
+ AC_REQUIRE([AC_PROG_SED])
+ AC_REQUIRE([AC_PROG_GREP])
+
+ AS_IF([test -n "$PYTHON"],[
+ ax_python_version="$1"
+
+ AC_MSG_CHECKING([for python version])
+ changequote(<<,>>)
+ python_version=`$PYTHON -V 2>&1 | $GREP "^Python " | $SED -e 's/^.* \([0-9]*\.[0-9]*\.[0-9]*\)/\1/'`
+ changequote([,])
+ AC_MSG_RESULT($python_version)
+
+ AC_SUBST([PYTHON_VERSION],[$python_version])
+
+ AX_COMPARE_VERSION([$ax_python_version],[le],[$python_version],[
+ :
+ $2
+ ],[
+ :
+ $3
+ ])
+ ],[
+ AC_MSG_WARN([could not find the python interpreter])
+ $3
+ ])
+])
diff --git a/autoconf-archive/ax_python_module.m4 b/autoconf-archive/ax_python_module.m4
new file mode 100644
index 0000000..f182c48
--- /dev/null
+++ b/autoconf-archive/ax_python_module.m4
@@ -0,0 +1,56 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_python_module.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PYTHON_MODULE(modname[, fatal, python])
+#
+# DESCRIPTION
+#
+# Checks for Python module.
+#
+# If fatal is non-empty then absence of a module will trigger an error.
+# The third parameter can either be "python" for Python 2 or "python3" for
+# Python 3; defaults to Python 3.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Andrew Collier
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 8
+
+AU_ALIAS([AC_PYTHON_MODULE], [AX_PYTHON_MODULE])
+AC_DEFUN([AX_PYTHON_MODULE],[
+ if test -z $PYTHON;
+ then
+ if test -z "$3";
+ then
+ PYTHON="python3"
+ else
+ PYTHON="$3"
+ fi
+ fi
+ PYTHON_NAME=`basename $PYTHON`
+ AC_MSG_CHECKING($PYTHON_NAME module: $1)
+ $PYTHON -c "import $1" 2>/dev/null
+ if test $? -eq 0;
+ then
+ AC_MSG_RESULT(yes)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=yes
+ else
+ AC_MSG_RESULT(no)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=no
+ #
+ if test -n "$2"
+ then
+ AC_MSG_ERROR(failed to find required module $1)
+ exit 1
+ fi
+ fi
+])
diff --git a/configure.ac b/configure.ac
index b611aca..ae7c11e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,6 +14,18 @@ AC_CONFIG_HEADER([config.h])
AC_CONFIG_SRCDIR([README])
AC_CONFIG_MACRO_DIR([m4])
+dnl Include some autoconf macros to check for python modules.
+dnl
+dnl These macros are coming from the autoconf archive at
+dnl http://www.gnu.org/software/autoconf-archive
+
+dnl This one is for the AX_PYTHON_MODULE() macro.
+m4_include([autoconf-archive/ax_python_module.m4])
+
+dnl These two below are for the AX_PROG_PYTHON_VERSION() module.
+m4_include([autoconf-archive/ax_compare_version.m4])
+m4_include([autoconf-archive/ax_prog_python_version.m4])
+
AM_INIT_AUTOMAKE([1.11.1 foreign subdir-objects tar-ustar parallel-tests])
AM_MAINTAINER_MODE([enable])
@@ -76,6 +88,12 @@ AC_ARG_ENABLE([bash-completion],
ENABLE_BASH_COMPLETION=$enableval,
ENABLE_BASH_COMPLETION=auto)
+AC_ARG_ENABLE([fedabipkgdiff],
+ AS_HELP_STRING([--enable-fedabipkgdiff=yes|no|auto],
+ [enable the fedabipkgdiff tool]),
+ ENABLE_FEDABIPKGDIFF=$enableval,
+ ENABLE_FEDABIPKGDIFF=auto)
+
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@@ -219,6 +237,68 @@ fi
AM_CONDITIONAL(ENABLE_BASH_COMPLETION, test x$ENABLE_BASH_COMPLETION = xyes)
+dnl if --enable-fedabipkgdiff has the 'auto' value, then check for the required
+dnl python modules. If they are present, then enable the fedabipkgdiff program.
+dnl If they are not then disable the program.
+dnl
+dnl If --enable-fedabipkgdiff has the 'yes' value, then check for the required
+dnl python modules and whatever dependency fedabipkgdiff needs. If they are
+dnl not present then the configure script will error out.
+
+if test x$ENABLE_FEDABIPKGDIFF = xauto -o x$ENABLE_FEDABIPKGDIFF = xyes; then
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=yes
+else
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=no
+fi
+
+if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
+ if test x$ENABLE_FEDABIPKGDIFF = xyes; then
+ FATAL=yes
+ fi
+
+ AC_PATH_PROG(WGET, wget, no)
+
+ if test x$WGET = x$no; then
+ AC_MSG_ERROR(could not find the wget program)
+ fi
+
+ # The minimal python version we want to support is 2.6.6 because EL6
+ # distributions have that version installed.
+ MINIMAL_PYTHON_VERSION="2.6.6"
+
+ AC_PATH_PROG(PYTHON, python, no)
+ AX_PROG_PYTHON_VERSION($MINIMAL_PYTHON_VERSION,
+ [MINIMAL_PYTHON_VERSION_FOUND=yes],
+ [MINIMAL_PYTHON_VERSION_FOUND=no])
+
+ if test x$MINIMAL_PYTHON_VERSION_FOUND = xno; then
+ AC_MSG_ERROR([could not find a python program of version at least $MINIMAL_PYTHON_VERSION])
+ fi
+
+ AX_PYTHON_MODULE(argparse, $FATAL, python2)
+ AX_PYTHON_MODULE(glob, $FATAL, python2)
+ AX_PYTHON_MODULE(logging, $FATAL, python2)
+ AX_PYTHON_MODULE(os, $FATAL, python2)
+ AX_PYTHON_MODULE(re, $FATAL, python2)
+ AX_PYTHON_MODULE(shlex, $FATAL, python2)
+ AX_PYTHON_MODULE(subprocess, $FATAL, python2)
+ AX_PYTHON_MODULE(sys, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(urlparse, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(shutil, $FATAL, python2)
+ AX_PYTHON_MODULE(unittest, $FATAL, python2)
+ AX_PYTHON_MODULE(koji, $FATAL, python2)
+ AX_PYTHON_MODULE(mock, $FATAL, python2)
+ ENABLE_FEDABIPKGDIFF=yes
+
+ if test x$ENABLE_FEDABIPKGDIFF != xyes; then
+ ENABLE_FEDABIPKGDIFF=no
+ fi
+fi
+
+AM_CONDITIONAL(ENABLE_FEDABIPKGDIFF, test x$ENABLE_FEDABIPKGDIFF = xyes)
+
dnl Check for dependency: libzip
LIBZIP_VERSION=0.10.1
@@ -361,7 +441,10 @@ libabigail.pc
bash-completion/Makefile])
dnl Some test scripts are generated by autofoo.
-AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh], [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh],
+ [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py],
+ [chmod +x tests/runtestfedabipkgdiff.py])
AC_OUTPUT
@@ -384,6 +467,7 @@ AC_MSG_NOTICE([
Enable deb support in abipkgdiff : ${ENABLE_DEB}
Enable GNU tar archive support in abipkgdiff : ${ENABLE_TAR}
Enable bash completion : ${ENABLE_BASH_COMPLETION}
+ Enable fedabipkgdiff : ${ENABLE_FEDABIPKGDIFF}
Generate html apidoc : ${ENABLE_APIDOC}
Generate html manual : ${ENABLE_MANUAL}
])
diff --git a/tests/Makefile.am b/tests/Makefile.am
index caf49e6..953dfef 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,9 +31,10 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
+runtestfedabipkgdiff.py \
$(CXX11_TESTS)
-EXTRA_DIST = runtestcanonicalizetypes.sh.in
+EXTRA_DIST = runtestcanonicalizetypes.sh.in runtestfedabipkgdiff.py.in
CLEANFILES = \
runtestcanonicalizetypes.output.txt \
runtestcanonicalizetypes.output.final.txt
@@ -114,6 +115,9 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
+runtestfedabipkgdiff_py_SOURCES =
+runtestfedabipkgdiff.py$(EXEEXT):
+
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
new file mode 100755
index 0000000..3087212
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -0,0 +1,450 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library. This program is free
+# software; you can redistribute it and/or modify it under the terms
+# of the GNU General Public License as published by the Free Software
+# Foundation; either version 3, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; see the file COPYING-LGPLV3. If
+# not, see <http://www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import os
+import itertools
+import shutil
+import unittest
+
+import koji
+
+try:
+ from mock import patch
+except ImportError:
+ print >>sys.stderr, \
+ 'mock is not installed. Please install it before running tests.'
+ sys.exit(1)
+
+import imp
+# Import the fedabipkgdiff program file from the source directory.
+fedabipkgdiff_mod = imp.load_source('fedabidiff',
+ '@top_srcdir@/tools/fedabipkgdiff')
+
+counter = itertools.count(0)
+
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_fedora_distro(self):
+ distro = 'fc5'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'f5'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc23'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc234'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7_2'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+
+class RPMTest(unittest.TestCase):
+ """Test case for RPM class"""
+
+ def setUp(self):
+ self.debuginfo_rpm_info = {
+ 'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+ self.rpm_info = {
+ 'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+
+ def test_attribute_access(self):
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertEquals(self.debuginfo_rpm_info['arch'], rpm.arch)
+ self.assertEquals(self.debuginfo_rpm_info['name'], rpm.name)
+ self.assertEquals(self.debuginfo_rpm_info['release'], rpm.release)
+ self.assertEquals(self.debuginfo_rpm_info['version'], rpm.version)
+
+ def test_raise_error_if_name_not_exist(self):
+ rpm = fedabipkgdiff_mod.RPM({})
+ try:
+ rpm.xxxxx
+ except AttributeError:
+ # Succeed, exit normally
+ return
+ self.fail('AttributeError should be raised, but not.')
+
+ def test_is_debuginfo(self):
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertTrue(rpm.is_debuginfo)
+
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertFalse(rpm.is_debuginfo)
+
+ def test_nvra(self):
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ nvra = koji.parse_NVRA(rpm.nvra)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ def test_str_representation(self):
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertEquals(str(self.rpm_info), str(rpm))
+
+
+class LocalRPMTest(unittest.TestCase):
+ """Test case for local RPM"""
+
+ def setUp(self):
+ self.filename = 'httpd-2.4.18-1.fc22.x86_64.rpm'
+
+ def test_file_parser(self):
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ full_filename = os.path.join('/', 'tmp', self.filename)
+ rpm = fedabipkgdiff_mod.LocalRPM(full_filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+ self.assertEquals(full_filename, rpm.downloaded_file)
+
+ @patch('os.path.exists')
+ def test_find_existent_debuginfo(self, mock_exists):
+ mock_exists.return_value = True
+
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertTrue(isinstance(rpm, fedabipkgdiff_mod.LocalRPM))
+
+ nvra = koji.parse_NVRA(self.filename)
+ expected_debuginfo = fedabipkgdiff_mod.LocalRPM(
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
+ debuginfo = rpm.find_debuginfo()
+ self.assertEquals(expected_debuginfo.name, debuginfo.name)
+ self.assertEquals(expected_debuginfo.version, debuginfo.version)
+ self.assertEquals(expected_debuginfo.release, debuginfo.release)
+
+ def test_find_non_existent_debuginfo(self):
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertEquals(None, rpm.find_debuginfo())
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff"""
+
+ def setUp(self):
+ self.pkg1_single_info = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ })
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_single_info = self.pkg1_single_info.copy()
+
+ self.pkg1_infos = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'x86_64': [
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'armv7hl': [
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_infos = self.pkg1_infos.copy()
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 0
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 4
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('fedabidiff.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+fake_rpm_file = 'foo-0.1-1.fc24.x86_64.rpm'
+
+
+class MockGlobalConfig(object):
+ koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ return MockKojiClientSession(baseurl=fedabipkgdiff_mod.DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock"""
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class SelectRpmsFromABuildTest(unittest.TestCase):
+ """Test case for select_rpms_from_a_build"""
+
+ def assert_rpms(self, rpms):
+ for item in rpms:
+ self.assertTrue(item.arch in ['i686', 'x86_64'])
+ self.assertTrue(item.name in ('httpd', 'httpd-debuginfo'))
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_all_arches(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabipkgdiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_one_arch(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabipkgdiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ session = fedabipkgdiff_mod.get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_cannot_find_a_latest_build_with_invalid_distro(self):
+ session = fedabipkgdiff_mod.get_session()
+ self.assertRaises(fedabipkgdiff_mod.NoCompleteBuilds,
+ session.get_package_latest_build, 'httpd', 'xxxx')
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('fedabidiff.koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ session = fedabipkgdiff_mod.get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/tools/Makefile.am b/tools/Makefile.am
index b855f41..0d96215 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -6,6 +6,12 @@ else
bin_PROGRAMS = abidiff abilint abidw abicompat abipkgdiff
endif
+if ENABLE_FEDABIPKGDIFF
+ bin_SCRIPTS = fedabipkgdiff
+else
+ noinst_SCRIPTS = fedabipkgdiff
+endif
+
noinst_PROGRAMS = abisym abinilint
if ENABLE_ZIP_ARCHIVE
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
new file mode 100755
index 0000000..4a5ba80
--- /dev/null
+++ b/tools/fedabipkgdiff
@@ -0,0 +1,805 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library. This program is free
+# software; you can redistribute it and/or modify it under the terms
+# of the GNU General Public License as published by the Free Software
+# Foundation; either version 3, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; see the file COPYING-LGPLV3. If
+# not, see <http://www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import argparse
+import logging
+import os
+import re
+import shlex
+import subprocess
+import sys
+
+from collections import namedtuple
+from itertools import groupby
+
+import koji
+
+
+DEFAULT_KOJI_SERVER = 'http://koji.fedoraproject.org/kojihub'
+DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
+
+HOME_DIR = os.path.join('/tmp',
+ os.path.splitext(os.path.basename(__file__))[0])
+
+# Used to construct abipkgdiff command line argument, package and associated
+# debuginfo package
+PkgInfo = namedtuple('PkgInfo', 'package debuginfo_package')
+
+
+global_config = None
+pathinfo = None
+session = None
+
+logging.basicConfig(format='[%(levelname)s] %(message)s',
+ level=logging.CRITICAL)
+logger = logging.getLogger(os.path.basename(__file__))
+
+
+class KojiPackageNotFound(Exception):
+ """Package is not found in Koji"""
+
+
+class PackageNotFound(Exception):
+ """Package is not found locally"""
+
+
+class RpmNotFound(Exception):
+ """RPM is not found"""
+
+
+class NoBuildsError(Exception):
+ """No builds returned from a method to select specific builds"""
+
+
+class NoCompleteBuilds(Exception):
+ """No complete builds for a package
+
+ This is a serious problem, nothing can be done if there is no complete
+ builds for a package.
+ """
+
+
+class InvalidDistroError(Exception):
+ """Invalid distro error"""
+
+
+class CannotFindLatestBuildError(Exception):
+ """Cannot find latest build from a package"""
+
+
+def is_distro_valid(distro):
+ """Adjust if a distro is valid
+
+ Currently, check for Fedora and RHEL.
+
+ :param str distro: a string representing a distro value.
+ :return: True if distro is the one specific to Fedora, like fc24, el7.
+ "rtype: bool
+ """
+ return re.match(r'^(fc|el)\d{1,2}$', distro) is not None
+
+
+def log_call(func):
+ def proxy(*args, **kwargs):
+ logger.debug('Call %s, args: %s, kwargs: %s',
+ func.__name__,
+ args if args else '',
+ kwargs if kwargs else '')
+ result = func(*args, **kwargs)
+ logger.debug('Result from %s: %s', func.__name__, result)
+ return result
+ return proxy
+
+
+class RPM(object):
+ """Represeting a RPM"""
+
+ def __init__(self, data):
+ """Initialize a RPM object
+
+ :param dict data: a dict representing a RPM information got from koji
+ API, either listRPMs or getRPM
+ """
+ self.data = data
+
+ def __str__(self):
+ return str(self.data)
+
+ def __getattr__(self, name):
+ if name in self.data:
+ return self.data[name]
+ else:
+ raise AttributeError('No attribute name {0}'.format(name))
+
+ @property
+ def nvra(self):
+ return '%(name)s-%(version)s-%(release)s.%(arch)s' % self.data
+
+ @property
+ def filename(self):
+ return '{0}.rpm'.format(self.nvra)
+
+ @property
+ def is_debuginfo(self):
+ """Check if a RPM is a debuginfo"""
+ return koji.is_debuginfo(self.data['name'])
+
+ @property
+ def download_url(self):
+ """Get the URL from where to download from koji"""
+ build = session.getBuild(self.build_id)
+ return os.path.join(pathinfo.build(build), pathinfo.rpm(self.data))
+
+ @property
+ def downloaded_file(self):
+ """Get a pridictable downloaded file name with absolute path"""
+ # arch should be removed from the result returned from PathInfo.rpm
+ filename = os.path.basename(pathinfo.rpm(self.data))
+ return os.path.join(get_download_dir(), filename)
+
+ @property
+ def is_downloaded(self):
+ return os.path.exists(self.downloaded_file)
+
+
+class LocalRPM(RPM):
+ """Representing a local RPM
+
+ Local RPM means the one that could be already downloaded or built from
+ where I can find it
+ """
+
+ def __init__(self, filename):
+ self.local_filename = filename
+ self.data = koji.parse_NVRA(os.path.basename(filename))
+
+ @property
+ def downloaded_file(self):
+ return self.local_filename
+
+ @property
+ def download_url(self):
+ raise NotImplementedError('LocalRPM has no URL to download')
+
+ @log_call
+ def find_debuginfo(self):
+ """Find debuginfo rpm package from a directory
+
+ :param str rpm_file: the rpm file name
+ :return: the absolute file name of the found debuginfo rpm
+ :rtype: str or None
+ """
+ search_dir = os.path.dirname(os.path.abspath(self.local_filename))
+ filename = \
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % \
+ self.data
+ filename = os.path.join(search_dir, filename)
+ return LocalRPM(filename) if os.path.exists(filename) else None
+
+
+class Brew(object):
+ """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff
+
+ kojihub XMLRPC APIs are well-documented in koji's source code. For more
+ details information, please refer to class RootExports within kojihub.py.
+ """
+
+ def __init__(self, baseurl):
+ """Initialize Brew that is a proxy to koji.ClientSession"""
+ self.session = koji.ClientSession(baseurl)
+
+ @log_call
+ def listRPMs(self, selector=None, **kwargs):
+ """Proxy to kojihub.listRPMs
+
+ :param selector: to adjust if a RPM should be selected
+ :type selector: a callable object
+ :param kwargs: keyword parameters accepted by kojihub.listRPMs
+ :type kwargs: dict
+ :return: a list of RPMs, each of them is a dict object
+ :rtype: list
+ """
+ if selector:
+ assert hasattr(selector, '__call__'), 'selector should be callable'
+ rpms = self.session.listRPMs(**kwargs)
+ if selector:
+ rpms = [rpm for rpm in rpms if selector(rpm)]
+ return rpms
+
+ @log_call
+ def getRPM(self, *args, **kwargs):
+ rpm = self.session.getRPM(*args, **kwargs)
+ if rpm is None:
+ raise RpmNotFound('Cannot find RPM {0}'.format(args[0]))
+ return rpm
+
+ @log_call
+ def listBuilds(self, topone=None, selector=None, order_by=None,
+ reverse=None, **kwargs):
+ """Proxy to kojihub.listBuilds to list completed builds
+
+ Suport additional two keyword parameters:
+
+ :param bool topone: whether to return the top first one
+ :param selector: a callable object used to select specific subset of
+ builds
+ :type selector: callable object
+ :param str order_by: the attribute name by which to order the builds,
+ for example, name, version, or nvr.
+ :param bool reverse: whether to order builds reversely
+ :param dict kwargs: keyword parameters accepted by kojihub.listBuilds
+ :return: a list of builds, even if just return only one build
+ :rtype: list
+ """
+ if 'state' not in kwargs:
+ kwargs['state'] = koji.BUILD_STATES['COMPLETE']
+
+ if selector is not None and not hasattr(selector, '__call__'):
+ raise TypeError(
+ '{0} is not a callable object.'.format(str(selector)))
+
+ if order_by is not None and not isinstance(order_by, basestring):
+ raise TypeError('order_by {0} is invalid.'.format(order_by))
+
+ builds = self.session.listBuilds(**kwargs)
+ if selector is not None:
+ builds = [build for build in builds if selector(build)]
+ if order_by is not None:
+ # FIXME: is it possible to sort builds by using opts parameter of
+ # listBuilds
+ builds = sorted(builds,
+ key=lambda item: item[order_by],
+ reverse=reverse)
+ if topone:
+ builds = builds[0:1]
+
+ return builds
+
+ @log_call
+ def getPackage(self, name):
+ """Proxy to kojihub.getPackage
+
+ :param str name: package name
+ :return: a dict object representing a package
+ :rtype: dict
+ """
+ package = self.session.getPackage(name)
+ if package is None:
+ package = self.session.getPackage(name.rsplit('-', 1)[0])
+ if package is None:
+ raise KojiPackageNotFound(
+ 'Cannot find package {0}.'.format(name))
+ return package
+
+ @log_call
+ def getBuild(self, *args, **kwargs):
+ """Proxy to kojihub.getBuild"""
+ return self.session.getBuild(*args, **kwargs)
+
+ @log_call
+ def get_rpm_build_id(self, name, version, release, arch=None):
+ """Get build ID that contains a rpm with specific nvra
+
+ If arch is omitted, a rpm can be identified easily by its N-V-R-A.
+
+ If arch is omitted, name is used to get associated package, and then
+ to get the build.
+
+ :param str name: name of a rpm
+ :param str version: version of a rpm
+ :param str release: release of a rpm
+ :param arch: arch of a rpm
+ :type arch: str or None
+ :return: the build from where the rpm is built
+ :rtype: dict
+ :raises KojiPackageNotFound: if name is not found from koji when arch
+ is None
+ """
+ if arch is None:
+ package = self.getPackage(name)
+ selector = lambda item: item['version'] == version and \
+ item['release'] == release
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector)
+ if not builds:
+ raise NoBuildsError(
+ 'No builds are selected from package {0}.'.format(
+ package['name']))
+ return builds[0]['build_id']
+ else:
+ rpm = self.getRPM({'name': name,
+ 'version': version,
+ 'release': release,
+ 'arch': arch,
+ })
+ return rpm['build_id']
+
+ @log_call
+ def get_package_latest_build(self, package_name, distro):
+ """Get latest build from a package
+
+ :param str package_name: from which package to get the latest build
+ :param str distro: which distro the latest build belongs to
+ :return: the found build
+ :rtype: dict or None
+ """
+ package = self.getPackage(package_name)
+ selector = lambda item: item['release'].find(distro) > -1
+
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector,
+ order_by='nvr',
+ reverse=True)
+ if not builds:
+ raise NoCompleteBuilds(
+ 'No complete builds of package {0}'.format(package_name))
+
+ return builds[0]
+
+ @log_call
+ def select_rpms_from_a_build(self, build_id, package_name, arches=None,
+ select_subpackages=None):
+ """Select specific RPMs within a build
+
+ rpms could be filtered be specific criterias by the parameters.
+
+ :param int build_id: from which build to select rpms.
+ :param str package_name: which rpm to select that matches this name.
+ :param arches: which arches to select. If arches omits, rpms with all
+ arches except noarch and src will be selected.
+ :type arches: list, tuple or None
+ :return: a list of rpms returned from listRPMs
+ :rtype: list
+ """
+ excluded_arches = ('noarch', 'src')
+
+ def rpms_selector(package_name, excluded_arches):
+ return lambda rpm: \
+ rpm['arch'] not in excluded_arches and \
+ (rpm['name'] == package_name or
+ rpm['name'].endswith('-debuginfo'))
+
+ if select_subpackages:
+ selector = lambda rpm: rpm['arch'] not in excluded_arches
+ else:
+ selector = rpms_selector(package_name, excluded_arches)
+ rpm_infos = self.listRPMs(buildID=build_id,
+ arches=arches,
+ selector=selector)
+ return [RPM(rpm_info) for rpm_info in rpm_infos]
+
+ @log_call
+ def get_latest_built_rpms(self, package_name, distro, arches=None):
+ """Get rpms from latest build of a package
+
+ By default, debuginfo rpm is also retrieved.
+
+ :param str package_name: from which package to get the rpms
+ :param str distro: which distro the rpms belong to
+ :param arches: which arches the rpms belong to
+ :type arches: str or None
+ :return: the selected rpms
+ :rtype: list
+ """
+ latest_build = self.get_package_latest_build(package_name, distro)
+ # Get rpm and debuginfo rpm from each arch
+ return self.select_rpms_from_a_build(latest_build['build_id'],
+ package_name,
+ arches=arches)
+
+
+@log_call
+def get_session():
+ return Brew(global_config.koji_server)
+
+
+@log_call
+def get_download_dir():
+ """Return the directory holding all downloaded rpms"""
+ download_dir = os.path.join(HOME_DIR, 'downloads')
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
+
+@log_call
+def download_rpm(url):
+ """Download a rpm"""
+ # TODO: wget is good, but there maybe a better way to do this
+ cmd = shlex.split('wget -q -P {0} -c {1}'.format(get_download_dir(), url))
+ proc = subprocess.Popen(cmd)
+ s_stdout, s_stderr = proc.communicate()
+ if proc.returncode > 0:
+ logger.error('wget fails. returned code: %d. message: %s',
+ proc.returncode, s_stderr)
+ return False
+ return True
+
+
+@log_call
+def download_rpms(pkg_info):
+ def _download(rpm):
+ if rpm.is_downloaded:
+ logger.debug('Reuse %s', rpm.downloaded_file)
+ else:
+ logger.debug('Download %s', rpm.download_url)
+ download_rpm(rpm.download_url)
+
+ for arch, rpm_infos in pkg_info.iteritems():
+ map(_download, rpm_infos)
+
+
+@log_call
+def abipkgdiff(pkg_info1, pkg_info2):
+ """Run abipkgdiff against found two RPM packages"""
+
+ cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ pkg_info1.debuginfo_package.downloaded_file,
+ pkg_info2.debuginfo_package.downloaded_file,
+ pkg_info1.package.downloaded_file,
+ pkg_info2.package.downloaded_file)
+
+ if global_config.dry_run:
+ print 'DRY-RUN:', cmd
+ return
+
+ logger.debug('Run: %s', cmd)
+
+ print 'ABI check on {0} and {1}'.format(pkg_info1.package.filename,
+ pkg_info2.package.filename)
+ print
+
+ proc = subprocess.Popen(shlex.split(cmd))
+ return proc.wait()
+
+
+def magic_construct(rpms):
+ """Construct RPMs into a magic structure
+
+ Convert list of
+
+ foo-1.0-1.fc22.i686
+ foo-debuginfo-1.0-1.fc22.i686
+ foo-devel-1.0-1.fc22.i686
+
+ to list of
+
+ (foo-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+ (foo-devel-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+ """
+ debuginfo = None
+ packages = []
+ for rpm in rpms:
+ if rpm.is_debuginfo:
+ debuginfo = rpm
+ else:
+ packages.append(rpm)
+ return [PkgInfo(package, debuginfo) for package in packages]
+
+
+@log_call
+def run_abipkgdiff(pkg1_infos, pkg2_infos):
+ """Run abipkgdiff
+
+ If one of the executions finds ABI differences, the return code is the
+ return code from abipkgdiff.
+
+ :param dict pkg1_infos: a dict mapping from arch to list of rpms, that is
+ returned from method make_rpms_usable_for_abipkgdiff
+ :return: exit code of the last non-zero returned from underlying abipkgdiff
+ :rtype: number
+ """
+ arches = pkg1_infos.keys()
+ arches.sort()
+
+ return_code = 0
+
+ for arch in arches:
+ pkg_infos = magic_construct(pkg1_infos[arch])
+
+ for pkg_info in pkg_infos:
+ rpms = pkg2_infos[arch]
+
+ package = [rpm for rpm in rpms
+ if rpm.name == pkg_info.package.name][0]
+ debuginfo = [rpm for rpm in rpms
+ if rpm.name == pkg_info.debuginfo_package.name][0]
+
+ ret = abipkgdiff(pkg_info,
+ PkgInfo(package=package,
+ debuginfo_package=debuginfo))
+ if ret > 0:
+ return_code = ret
+
+ return return_code
+
+
+@log_call
+def diff_local_rpm_with_latest_rpm_from_koji():
+ """Diff against local rpm and remove latest rpm
+
+ This operation handles a local rpm and debuginfo rpm and remote ones
+ located in remote Koji server, that has specific distro specificed by
+ argument --from.
+
+ 1/ Suppose the packager has just locally built a package named
+ foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
+ latest stable package from Fedora 23, one would do:
+
+ fedabipkgdiff --from f23 ./foo-3.0.fc24.rpm
+ """
+
+ from_distro = global_config.from_distro
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ local_rpm_file = global_config.NVR[0]
+ if not os.path.exists(local_rpm_file):
+ raise ValueError('{0} does not exist.'.format(local_rpm_file))
+
+ local_rpm = LocalRPM(local_rpm_file)
+ local_debuginfo = local_rpm.find_debuginfo()
+ if local_debuginfo is None:
+ raise ValueError(
+ 'debuginfo rpm {0} does not exist.'.format(local_debuginfo))
+
+ rpms = session.get_latest_built_rpms(local_rpm.name,
+ from_distro,
+ arches=local_rpm.arch)
+ pkg_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg_infos)
+
+ rpms = pkg_infos.values()[0]
+ package, debuginfo = sorted(rpms, key=lambda rpm: rpm.name)
+ return abipkgdiff(PkgInfo(package, debuginfo),
+ PkgInfo(local_rpm, local_debuginfo))
+
+
+@log_call
+def make_rpms_usable_for_abipkgdiff(rpms):
+ """
+ Construct result that contains mappings from arch to download url and
+ downloaded rpm filename of rpm and debuginfo rpm
+
+ :return: a mapping from an arch to a list of rpms
+ :rtype: dict
+ """
+ result = {}
+ rpms_iter = groupby(sorted(rpms, key=lambda rpm: rpm.arch),
+ key=lambda item: item.arch)
+ for arch, rpms in rpms_iter:
+ result[arch] = list(rpms)
+ return result
+
+
+@log_call
+def diff_latest_rpms_based_on_distros():
+ """abipkgdiff rpms based on two distros
+
+ 2/ Suppose the packager wants to see how the ABIs of the package foo
+ evolved between fedora 19 and fedora 22. She would thus type the command:
+
+ fedabipkgdiff --from f19 --to f22 foo
+ """
+
+ from_distro = global_config.from_distro
+ to_distro = global_config.to_distro
+
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ if not is_distro_valid(to_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(distro))
+
+ package_name = global_config.NVR[0]
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.from_distro)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg1_infos)
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.to_distro)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+@log_call
+def diff_rpms_with_nvra(name, version, release, arch=None,
+ all_subpackages=None):
+ build_id = session.get_rpm_build_id(name, version, release, arch)
+ rpms = session.select_rpms_from_a_build(build_id, name, arches=arch,
+ select_subpackages=all_subpackages)
+ return make_rpms_usable_for_abipkgdiff(rpms)
+
+
+@log_call
+def diff_two_nvras_from_koji():
+ """Diff two nvras from koji
+
+ The arch probably omits, that means febabipkgdiff will diff all arches. If
+ specificed, the specific arch will be handled.
+
+ 3/ Suppose the packager wants to compare the ABI of two packages designated
+ by their name and version. She would issue a command like this:
+
+ fedabipkgdiff foo-1.0.fc19 foo-3.0.fc24
+ fedabipkgdiff foo-1.0.fc19.i686 foo-1.0.fc24.i686
+ """
+ left_rpm = koji.parse_NVRA(global_config.NVR[0])
+ right_rpm = koji.parse_NVRA(global_config.NVR[1])
+
+ if is_distro_valid(left_rpm['arch']) and \
+ is_distro_valid(right_rpm['arch']):
+ nvr = koji.parse_NVR(global_config.NVR[0])
+ params1 = (nvr['name'], nvr['version'], nvr['release'])
+
+ nvr = koji.parse_NVR(global_config.NVR[1])
+ params2 = (nvr['name'], nvr['version'], nvr['release'])
+ else:
+ params1 = (left_rpm['name'],
+ left_rpm['version'],
+ left_rpm['release'],
+ left_rpm['arch'])
+ params2 = (right_rpm['name'],
+ right_rpm['version'],
+ right_rpm['release'],
+ right_rpm['arch'])
+
+ pkg1_infos = diff_rpms_with_nvra(
+ *params1, all_subpackages=global_config.check_all_subpackages)
+ download_rpms(pkg1_infos)
+
+ pkg2_infos = diff_rpms_with_nvra(
+ *params2, all_subpackages=global_config.check_all_subpackages)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def build_commandline_args_parser():
+ parser = argparse.ArgumentParser(
+ description='Run abipkgdiff against RPM packages from koji')
+
+ parser.add_argument(
+ 'NVR',
+ nargs='*',
+ help='RPM package N-V-R, N-V-R-A, N, or a local RPM '
+ 'file name with relative or absolute path.')
+ parser.add_argument(
+ '--dry-run',
+ required=False,
+ dest='dry_run',
+ action='store_true',
+ help='Don\'t actually run abipkgdiff. The commands that should be '
+ 'run will be sent to stdout.')
+ parser.add_argument(
+ '--from',
+ required=False,
+ metavar='DISTRO',
+ dest='from_distro',
+ help='baseline Fedora distro, for example, fc23')
+ parser.add_argument(
+ '--to',
+ required=False,
+ metavar='DISTRO',
+ dest='to_distro',
+ help='which Fedora distro to compare, for example, fc24')
+ parser.add_argument(
+ '-a',
+ '--all-subpackages',
+ required=False,
+ action='store_true',
+ dest='check_all_subpackages',
+ help='Check all subpackages instead of only the package specificed in '
+ 'command line.')
+ parser.add_argument(
+ '--debug',
+ required=False,
+ action='store_true',
+ dest='debug',
+ help='show debug output')
+ parser.add_argument(
+ '--traceback',
+ required=False,
+ action='store_true',
+ dest='show_traceback',
+ help='show traceback when there is an exception thrown.')
+ parser.add_argument(
+ '--server',
+ required=False,
+ metavar='URL',
+ dest='koji_server',
+ default=DEFAULT_KOJI_SERVER,
+ help='URL of koji XMLRPC service. Default is {0}'.format(
+ DEFAULT_KOJI_SERVER))
+ parser.add_argument(
+ '--topdir',
+ required=False,
+ metavar='URL',
+ dest='koji_topdir',
+ default=DEFAULT_KOJI_TOPDIR,
+ help='URL for RPM files access')
+
+ return parser
+
+
+def main():
+ parser = build_commandline_args_parser()
+
+ args = parser.parse_args()
+
+ global global_config
+ global_config = args
+
+ global pathinfo
+ pathinfo = koji.PathInfo(topdir=global_config.koji_topdir)
+
+ global session
+ session = get_session()
+
+ if global_config.debug:
+ logger.setLevel(logging.DEBUG)
+
+ logger.debug(args)
+
+ if global_config.from_distro and global_config.to_distro is None and \
+ global_config.NVR:
+ returncode = diff_local_rpm_with_latest_rpm_from_koji()
+
+ elif global_config.from_distro and global_config.to_distro and \
+ global_config.NVR:
+ returncode = diff_latest_rpms_based_on_distros()
+
+ elif global_config.from_distro is None and \
+ global_config.to_distro is None and len(global_config.NVR) > 1:
+ returncode = diff_two_nvras_from_koji()
+
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to -h.'
+ returncode = 1
+
+ return returncode
+
+
+if __name__ == '__main__':
+ try:
+ main()
+ except KeyboardInterrupt:
+ if global_config.debug:
+ logger.debug('Terminate by user')
+ else:
+ print >>sys.stderr, 'Terminate by user'
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(2)
+ except Exception as e:
+ if global_config.debug:
+ logger.debug(str(e))
+ else:
+ print >>sys.stderr, str(e)
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(1)
--
2.5.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Chenxiong Qi
@ 2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Fix adding fedabipkgdiff to source distribution tarball Dodji Seketeli
0 siblings, 2 replies; 10+ messages in thread
From: Dodji Seketeli @ 2016-01-01 0:00 UTC (permalink / raw)
To: Chenxiong Qi; +Cc: libabigail
[-- Attachment #1: Type: text/plain, Size: 9043 bytes --]
Hello Chenxiong,
Chenxiong Qi <cqi@redhat.com> writes:
> An updated patch with a small fix.
Okay, this is the one I have looked at. Thanks.
>> This is a new tool fedabipkgdiff that would be much convenient for
>> Fedora packagers to check potential ABI/API differences quickly using
>> abipkgdiff shipped with libabigail. This tool came from a cool idea from
>> Dodji.
[...]
>> For more details, please refer to
>> https://sourceware.org/bugzilla/show_bug.cgi?id=19428
Thank you very much for hacking on this enhancement request.
>> Currently, as the first step, it supports following ways,
>>
>> fedabipkgdiff --from fc23 ./foo-0.1-1.fc23.x86_64.rpm
>> fedabipkgdiff --from fc23 --to fc24 foo
>> fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
>> fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
This is really cool. Seriously. I have tested the patch and it works
for me :-)
>> Next step is to support the 4th use case mentioned in bug 19428.
Great.
>> fedabipkgdiff is being under development, still need to improve. Welcome
>> any feedback.
I do have some comments, please find them below.
> Subject: [PATCH] new tool of fedabipkgdiff
>
> fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
> compatibility issues quickly.
>
> Currently with the first version of fedabipkgdiff, you can invoke it in
> following ways.
>
> fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
> fedabipkgdiff --from fc23 --to fc24 foo
> fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
> fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
>
> Bug 19428
The commit log of patches must follow a format that is explained in the
source tree. You can read it online at
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=blob_plain;f=COMMIT-LOG-GUIDELINES.
This patch is missing the ChangeLog part in the GNU style format. I can
add it later for you if you like, as an example :-)
> .gitignore | 3 +
> tests/Makefile.am | 4 +
> tests/runtestfedabipkgdiff.sh | 5 +
> tools/fedabipkgdiff | 855 ++++++++++++++++++++++++++++++++++++++++++
I see that the patch is missing some changes necessary to integrate it
to the autotool machinery and have the 'configure' script check for the
dependencies of the fedabipkgdiff tool (mostly all the python modules
and whatnot). I know this can look dreadful at first sight, so I made
the changes. I have attached the patch at the end of this message.
After those changes, the output of the configure script (that's related
to my changes looks like):
checking for wget... /bin/wget
checking for python... /bin/python
checking for python version... 2.7.5
checking python module: argparse... yes
checking python module: glob... yes
checking python module: logging... yes
checking python module: os... yes
checking python module: re... yes
checking python module: shlex... yes
checking python module: subprocess... yes
checking python module: sys... yes
checking python module: itertools... yes
checking python module: urlparse... yes
checking python module: koji... yes
[...]
configure:
=====================================================================
Libabigail: 1.0.rc3
=====================================================================
Here is the configuration of the package:
Prefix : /home/dodji/.local
Source code location : /home/dodji/git/libabigail/fedabipkgdiff
C Compiler : gcc
C++ Compiler : g++
OPTIONAL FEATURES:
Enable zip archives : no
Use a C++-11 compiler : no
Enable rpm support in abipkgdiff : yes
Enable deb support in abipkgdiff : yes
Enable GNU tar archive support in abipkgdiff : yes
Enable bash completion : auto
Enable fedabipkgdiff : yes
Generate html apidoc : yes
Generate html manual : yes
(see how there is now a new line 'Enable fedabipkgdiff : yes' in the
final report).
For the record, I have put the patch, along with yours into a private
branch named "dodji/fedabipkgdiff" at
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=shortlog;h=refs/heads/dodji/fedabipkgdiff.
[...]
I guess you can just incorporate that patch inside your next version,
unless you'd want me to change something. Please tell me :-)
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index caf49e6..958995c 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -31,6 +31,7 @@ runtestlookupsyms \
> runtestaltdwarf \
> runtestcorediff \
> runtestabidiffexit \
> +runtestfedabipkgdiff.sh \
> $(CXX11_TESTS)
[...]
> diff --git a/tests/runtestfedabipkgdiff.sh b/tests/runtestfedabipkgdiff.sh
> new file mode 100755
> index 0000000..4144419
> --- /dev/null
> +++ b/tests/runtestfedabipkgdiff.sh
> @@ -0,0 +1,5 @@
> +#!/usr/bin/bash
> +
> +export FEDABIPKGDIFF_TESTS=1
> +
> +../../tools/fedabipkgdiff
> \ No newline at end of file
Actually, I think the runtestfedabipkgdiff.sh could be a proper python
script that contains the actual unit tests of fedabipkgdiff, rather than
having the unit tests be in the fedabipkgdiff file itself.
That file would then load (using the imp.load_source() function from the
python standard library) by specifying the path to the the fedabipkgdiff
from the source, so that "make check" uses the fedabipkgdiff from the
sources and not the one that might already be installed on the
system. There is a potentially a little bit of autoconf magic to do
here, but I can help you here for the details.
[...]
> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
> new file mode 100755
> index 0000000..5d62db2
> --- /dev/null
> +++ b/tools/fedabipkgdiff
> @@ -0,0 +1,855 @@
> +#!/usr/bin/env python
> +# -*- coding: utf-8 -*-
> +
> +import argparse
> +import glob
p> +import logging
> +import os
> +import re
> +import shlex
> +import subprocess
> +import sys
> +
> +from itertools import groupby
> +from urlparse import urlparse
> +
> +import koji
> +
> +"""
> +Find proper RPM packages from koji to run abipkgdiff.
> +
> +Internal structure is
> +
> +fc23 fc24
> +i686 i686
> + foo-0.1-1.fc23.i686.rpm foo-0.2-1.fc24.i686.rpm
> + foo-debuginfo-0.1-1.fc23.i686.rpm foo-debuginfo-0.2-1.fc23.i686.rpm
> +x86_64 x86_64
> + foo-0.1-1.fc23.x86_64.rpm foo-0.2-1.fc24.x86_64.rpm
> + foo-debuginfo-0.1-1.fc23.x86_64.rpm
> foo-debuginfo-0.2-1.fc23.x86_64.rpm
This is the 'internal structure' of what exactly? I guess it's the
structure of the cache where the packages are put after they are
downloaded. It'd be cool to be explicit here and give more details so
that readers won't have to guess.
[...]
OK, I am skipping some details. I think we'll have more opportunity to
refine them later :-) For now, I am focusing on things that I believe
belong to the big picture.
> +class Brew(object):
> + """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff"""
It looks like the kojihub XMLRPC API is not that well documented, or is
it? Maybe you could provide a link in the comments about where the APIs
are documented.
> +
> + def __init__(self, baseurl):
> + self.session = koji.ClientSession(baseurl)
> +
> + def listRPMs(self, **kwargs):
It'd be nice to detail the exact set of parameters expected here, their
type, and their meaning, even if those are supposed to be
declared/defined in the Kojihub API. As that API is not defined here,
it's important that we write down what we think we expect. I think we
ought to do that generally for all functions, especially those that
expect variable arguments or worse, keyword arguments like this. It's
important that readers knows what the function expects.
[...]
> +if 'FEDABIPKGDIFF_TESTS' not in os.environ and invoked_from_cmd:
> + try:
> + sys.exit(main())
> + except Exception as e:
> + print >>sys.stderr, str(e)
> + sys.exit(1)
> +
> +
> +import itertools
> +import shutil
> +import unittest
> +
As I was saying, earlier, I think this part should be split out from
this file and put in the separate regression test file under tests/.
Hmhm, I guess I should add these to the set of pre-requisite modules
checked by the configure script too ...
Other than that, I think this is an excellent start. This thing is
really cool. Thank you thank you thank you for tackling this. I find
this *super cool*. Not having to download the packages for which we
want to compare the ABI? this is pure magic ... I am so thrilled :-)
> Happy Hacking
Indeed, I am so happy right now :-)
Thanks!
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Better autotool integration for fedabipkgdiff --]
[-- Type: text/x-patch, Size: 18700 bytes --]
From 35bbdc075b95c69b9e801f9f45ff4d5b14ebbad1 Mon Sep 17 00:00:00 2001
From: Dodji Seketeli <dodji@redhat.com>
Date: Fri, 19 Feb 2016 17:38:16 +0100
Subject: [PATCH 2/2] Add autotooling to fedabipkgdiff
This patch adds detection for the dependencies of fedabipkgdiff. To
do that I had to use some extra autoconf macros from the GNU Autoconf
Archive at http://www.gnu.org/software/autoconf-archive. I got the
macros from the tarball of the autoconf-archive at
http://mirror0.babylon.network/gnu/autoconf-archive/autoconf-archive-2015.09.25.tar.xz.
Basically, the patch adds a new --enable-fedabipkgdiff option to the
configure program. By default the value of that option is "auto".
In that "auto" mode, the configure program checks for the dependencies
of fedabipkgdiff. If they are present then the tool will be installed
by 'make install'. If they are not, then the tool will just be
disabled and 'make install' won't install it.
If --enable-fedabipkgdiff is explicitly provided, then it's value is
set to 'yes'. In that mode, the configure program checks for the
dependencies of fedabipkgdiff too. If any of the dependencies is not
found on the system, the configure program fails and requires the
users to install the missing dependency.
The patch augments the report that is emitted at the end of the
configure script, and make it show the status of the "fedabipkgdiff"
feature; that is, it says if fedabipkgdiff is enabled or not.
The patch also ensures that the new fedabipkgdiff is included to the
source distribution that is built by the standard "make dist" command.
* autoconf-archive/ax_compare_version.m4: New file copied from the
autoconf-archive project.
* autoconf-archive/ax_prog_python_version.m4: Likewise.
* autoconf-archive/ax_python_module.m4: Likewise.
* Makefile.am: Add the new files above to the source distribution.
* configure.ac: Include the new m4 macros from the autoconf
archive. Add a new --enable-fedabipkgdiff option. Update the
report at the end of the configure process to show the status of
the fedabipkgdiff feature.
* tools/Makefile.am: Include the fedabipkgdiff to the source
distribution and install it if the "fedabipkgdiff" feature is
enabled.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
Makefile.am | 3 +
autoconf-archive/ax_compare_version.m4 | 177 +++++++++++++++++++++++++++++
autoconf-archive/ax_prog_python_version.m4 | 66 +++++++++++
autoconf-archive/ax_python_module.m4 | 56 +++++++++
configure.ac | 77 +++++++++++++
tools/Makefile.am | 6 +
6 files changed, 385 insertions(+)
create mode 100644 autoconf-archive/ax_compare_version.m4
create mode 100644 autoconf-archive/ax_prog_python_version.m4
create mode 100644 autoconf-archive/ax_python_module.m4
diff --git a/Makefile.am b/Makefile.am
index c855cf6..1ae2290 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -12,6 +12,9 @@ pkgconfig_DATA = libabigail.pc
#dist_bashcompletion_DATA =
EXTRA_DIST = \
+autoconf-archive/ax_python_module.m4 \
+autoconf-archive/ax_prog_python_version.m4 \
+autoconf-archive/ax_compare_version.m4 \
NEWS README COPYING ChangeLog \
COPYING-LGPLV2 COPYING-LGPLV3 \
COPYING-GPLV3 gen-changelog.py \
diff --git a/autoconf-archive/ax_compare_version.m4 b/autoconf-archive/ax_compare_version.m4
new file mode 100644
index 0000000..74dc0fd
--- /dev/null
+++ b/autoconf-archive/ax_compare_version.m4
@@ -0,0 +1,177 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_compare_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_COMPARE_VERSION(VERSION_A, OP, VERSION_B, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# This macro compares two version strings. Due to the various number of
+# minor-version numbers that can exist, and the fact that string
+# comparisons are not compatible with numeric comparisons, this is not
+# necessarily trivial to do in a autoconf script. This macro makes doing
+# these comparisons easy.
+#
+# The six basic comparisons are available, as well as checking equality
+# limited to a certain number of minor-version levels.
+#
+# The operator OP determines what type of comparison to do, and can be one
+# of:
+#
+# eq - equal (test A == B)
+# ne - not equal (test A != B)
+# le - less than or equal (test A <= B)
+# ge - greater than or equal (test A >= B)
+# lt - less than (test A < B)
+# gt - greater than (test A > B)
+#
+# Additionally, the eq and ne operator can have a number after it to limit
+# the test to that number of minor versions.
+#
+# eq0 - equal up to the length of the shorter version
+# ne0 - not equal up to the length of the shorter version
+# eqN - equal up to N sub-version levels
+# neN - not equal up to N sub-version levels
+#
+# When the condition is true, shell commands ACTION-IF-TRUE are run,
+# otherwise shell commands ACTION-IF-FALSE are run. The environment
+# variable 'ax_compare_version' is always set to either 'true' or 'false'
+# as well.
+#
+# Examples:
+#
+# AX_COMPARE_VERSION([3.15.7],[lt],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[lt],[3.15.8])
+#
+# would both be true.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[gt],[3.15.8])
+#
+# would both be false.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq2],[3.15.8])
+#
+# would be true because it is only comparing two minor versions.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq0],[3.15])
+#
+# would be true because it is only comparing the lesser number of minor
+# versions of the two values.
+#
+# Note: The characters that separate the version numbers do not matter. An
+# empty string is the same as version 0. OP is evaluated by autoconf, not
+# configure, so must be a string, not a variable.
+#
+# The author would like to acknowledge Guido Draheim whose advice about
+# the m4_case and m4_ifvaln functions make this macro only include the
+# portions necessary to perform the specific comparison specified by the
+# OP argument in the final configure script.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Tim Toolan <toolan@ele.uri.edu>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+dnl #########################################################################
+AC_DEFUN([AX_COMPARE_VERSION], [
+ AC_REQUIRE([AC_PROG_AWK])
+
+ # Used to indicate true or false condition
+ ax_compare_version=false
+
+ # Convert the two version strings to be compared into a format that
+ # allows a simple string comparison. The end result is that a version
+ # string of the form 1.12.5-r617 will be converted to the form
+ # 0001001200050617. In other words, each number is zero padded to four
+ # digits, and non digits are removed.
+ AS_VAR_PUSHDEF([A],[ax_compare_version_A])
+ A=`echo "$1" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ AS_VAR_PUSHDEF([B],[ax_compare_version_B])
+ B=`echo "$3" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ dnl # In the case of le, ge, lt, and gt, the strings are sorted as necessary
+ dnl # then the first line is used to determine if the condition is true.
+ dnl # The sed right after the echo is to remove any indented white space.
+ m4_case(m4_tolower($2),
+ [lt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [gt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [le],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],
+ [ge],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],[
+ dnl Split the operator from the subversion count if present.
+ m4_bmatch(m4_substr($2,2),
+ [0],[
+ # A count of zero means use the length of the shorter version.
+ # Determine the number of characters in A and B.
+ ax_compare_version_len_A=`echo "$A" | $AWK '{print(length)}'`
+ ax_compare_version_len_B=`echo "$B" | $AWK '{print(length)}'`
+
+ # Set A to no more than B's length and B to no more than A's length.
+ A=`echo "$A" | sed "s/\(.\{$ax_compare_version_len_B\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(.\{$ax_compare_version_len_A\}\).*/\1/"`
+ ],
+ [[0-9]+],[
+ # A count greater than zero means use only that many subversions
+ A=`echo "$A" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ ],
+ [.+],[
+ AC_WARNING(
+ [illegal OP numeric parameter: $2])
+ ],[])
+
+ # Pad zeros at end of numbers to make same length.
+ ax_compare_version_tmp_A="$A`echo $B | sed 's/./0/g'`"
+ B="$B`echo $A | sed 's/./0/g'`"
+ A="$ax_compare_version_tmp_A"
+
+ # Check for equality or inequality as necessary.
+ m4_case(m4_tolower(m4_substr($2,0,2)),
+ [eq],[
+ test "x$A" = "x$B" && ax_compare_version=true
+ ],
+ [ne],[
+ test "x$A" != "x$B" && ax_compare_version=true
+ ],[
+ AC_WARNING([illegal OP parameter: $2])
+ ])
+ ])
+
+ AS_VAR_POPDEF([A])dnl
+ AS_VAR_POPDEF([B])dnl
+
+ dnl # Execute ACTION-IF-TRUE / ACTION-IF-FALSE.
+ if test "$ax_compare_version" = "true" ; then
+ m4_ifvaln([$4],[$4],[:])dnl
+ m4_ifvaln([$5],[else $5])dnl
+ fi
+]) dnl AX_COMPARE_VERSION
diff --git a/autoconf-archive/ax_prog_python_version.m4 b/autoconf-archive/ax_prog_python_version.m4
new file mode 100644
index 0000000..628a3e4
--- /dev/null
+++ b/autoconf-archive/ax_prog_python_version.m4
@@ -0,0 +1,66 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_prog_python_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PROG_PYTHON_VERSION([VERSION],[ACTION-IF-TRUE],[ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# Makes sure that python supports the version indicated. If true the shell
+# commands in ACTION-IF-TRUE are executed. If not the shell commands in
+# ACTION-IF-FALSE are run. Note if $PYTHON is not set (for example by
+# running AC_CHECK_PROG or AC_PATH_PROG) the macro will fail.
+#
+# Example:
+#
+# AC_PATH_PROG([PYTHON],[python])
+# AX_PROG_PYTHON_VERSION([2.4.4],[ ... ],[ ... ])
+#
+# This will check to make sure that the python you have supports at least
+# version 2.4.4.
+#
+# NOTE: This macro uses the $PYTHON variable to perform the check.
+# AX_WITH_PYTHON can be used to set that variable prior to running this
+# macro. The $PYTHON_VERSION variable will be valorized with the detected
+# version.
+#
+# LICENSE
+#
+# Copyright (c) 2009 Francesco Salvestrini <salvestrini@users.sourceforge.net>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+AC_DEFUN([AX_PROG_PYTHON_VERSION],[
+ AC_REQUIRE([AC_PROG_SED])
+ AC_REQUIRE([AC_PROG_GREP])
+
+ AS_IF([test -n "$PYTHON"],[
+ ax_python_version="$1"
+
+ AC_MSG_CHECKING([for python version])
+ changequote(<<,>>)
+ python_version=`$PYTHON -V 2>&1 | $GREP "^Python " | $SED -e 's/^.* \([0-9]*\.[0-9]*\.[0-9]*\)/\1/'`
+ changequote([,])
+ AC_MSG_RESULT($python_version)
+
+ AC_SUBST([PYTHON_VERSION],[$python_version])
+
+ AX_COMPARE_VERSION([$ax_python_version],[le],[$python_version],[
+ :
+ $2
+ ],[
+ :
+ $3
+ ])
+ ],[
+ AC_MSG_WARN([could not find the python interpreter])
+ $3
+ ])
+])
diff --git a/autoconf-archive/ax_python_module.m4 b/autoconf-archive/ax_python_module.m4
new file mode 100644
index 0000000..f182c48
--- /dev/null
+++ b/autoconf-archive/ax_python_module.m4
@@ -0,0 +1,56 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_python_module.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PYTHON_MODULE(modname[, fatal, python])
+#
+# DESCRIPTION
+#
+# Checks for Python module.
+#
+# If fatal is non-empty then absence of a module will trigger an error.
+# The third parameter can either be "python" for Python 2 or "python3" for
+# Python 3; defaults to Python 3.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Andrew Collier
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 8
+
+AU_ALIAS([AC_PYTHON_MODULE], [AX_PYTHON_MODULE])
+AC_DEFUN([AX_PYTHON_MODULE],[
+ if test -z $PYTHON;
+ then
+ if test -z "$3";
+ then
+ PYTHON="python3"
+ else
+ PYTHON="$3"
+ fi
+ fi
+ PYTHON_NAME=`basename $PYTHON`
+ AC_MSG_CHECKING($PYTHON_NAME module: $1)
+ $PYTHON -c "import $1" 2>/dev/null
+ if test $? -eq 0;
+ then
+ AC_MSG_RESULT(yes)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=yes
+ else
+ AC_MSG_RESULT(no)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=no
+ #
+ if test -n "$2"
+ then
+ AC_MSG_ERROR(failed to find required module $1)
+ exit 1
+ fi
+ fi
+])
diff --git a/configure.ac b/configure.ac
index f7efca5..687a2ca 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,6 +14,18 @@ AC_CONFIG_HEADER([config.h])
AC_CONFIG_SRCDIR([README])
AC_CONFIG_MACRO_DIR([m4])
+dnl Include some autoconf macros to check for python modules.
+dnl
+dnl These macros are coming from the autoconf archive at
+dnl http://www.gnu.org/software/autoconf-archive
+
+dnl This one is for the AX_PYTHON_MODULE() macro.
+m4_include([autoconf-archive/ax_python_module.m4])
+
+dnl These two below are for the AX_PROG_PYTHON_VERSION() module.
+m4_include([autoconf-archive/ax_compare_version.m4])
+m4_include([autoconf-archive/ax_prog_python_version.m4])
+
AM_INIT_AUTOMAKE([1.11.1 foreign subdir-objects tar-ustar parallel-tests])
AM_MAINTAINER_MODE([enable])
@@ -76,6 +88,12 @@ AC_ARG_ENABLE([bash-completion],
ENABLE_BASH_COMPLETION=$enableval,
ENABLE_BASH_COMPLETION=auto)
+AC_ARG_ENABLE([fedabipkgdiff],
+ AS_HELP_STRING([--enable-fedabipkgdiff=yes|no|auto],
+ [enable the fedabipkgdiff tool]),
+ ENABLE_FEDABIPKGDIFF=$enableval,
+ ENABLE_FEDABIPKGDIFF=auto)
+
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@@ -219,6 +237,64 @@ fi
AM_CONDITIONAL(ENABLE_BASH_COMPLETION, test x$ENABLE_BASH_COMPLETION = xyes)
+dnl if --enable-fedabipkgdiff has the 'auto' value, then check for the required
+dnl python modules. If they are present, then enable the fedabipkgdiff program.
+dnl If they are not then disable the program.
+dnl
+dnl If --enable-fedabipkgdiff has the 'yes' value, then check for the required
+dnl python modules and whatever dependency fedabipkgdiff needs. If they are
+dnl not present then the configure script will error out.
+
+if test x$ENABLE_FEDABIPKGDIFF = xauto -o x$ENABLE_FEDABIPKGDIFF = xyes; then
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=yes
+else
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=no
+fi
+
+if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
+ if test x$ENABLE_FEDABIPKGDIFF = xyes; then
+ FATAL=yes
+ fi
+
+ AC_PATH_PROG(WGET, wget, no)
+
+ if test x$WGET = x$no; then
+ AC_MSG_ERROR(could not find the wget program)
+ fi
+
+ # The minimal python version we want to support is 2.6.6 because EL6
+ # distributions have that version installed.
+ MINIMAL_PYTHON_VERSION="2.6.6"
+
+ AC_PATH_PROG(PYTHON, python, no)
+ AX_PROG_PYTHON_VERSION($MINIMAL_PYTHON_VERSION,
+ [MINIMAL_PYTHON_VERSION_FOUND=yes],
+ [MINIMAL_PYTHON_VERSION_FOUND=no])
+
+ if test x$MINIMAL_PYTHON_VERSION_FOUND = xno; then
+ AC_MSG_ERROR([could not find a python program of version at least $MINIMAL_PYTHON_VERSION])
+ fi
+
+ AX_PYTHON_MODULE(argparse, $FATAL, python2)
+ AX_PYTHON_MODULE(glob, $FATAL, python2)
+ AX_PYTHON_MODULE(logging, $FATAL, python2)
+ AX_PYTHON_MODULE(os, $FATAL, python2)
+ AX_PYTHON_MODULE(re, $FATAL, python2)
+ AX_PYTHON_MODULE(shlex, $FATAL, python2)
+ AX_PYTHON_MODULE(subprocess, $FATAL, python2)
+ AX_PYTHON_MODULE(sys, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(urlparse, $FATAL, python2)
+ AX_PYTHON_MODULE(koji, $FATAL, python2)
+ ENABLE_FEDABIPKGDIFF=yes
+
+ if test x$ENABLE_FEDABIPKGDIFF != xyes; then
+ ENABLE_FEDABIPKGDIFF=no
+ fi
+fi
+
+AM_CONDITIONAL(ENABLE_FEDABIPKGDIFF, test x$ENABLE_FEDABIPKGDIFF = xyes)
+
dnl Check for dependency: libzip
LIBZIP_VERSION=0.10.1
@@ -384,6 +460,7 @@ AC_MSG_NOTICE([
Enable deb support in abipkgdiff : ${ENABLE_DEB}
Enable GNU tar archive support in abipkgdiff : ${ENABLE_TAR}
Enable bash completion : ${ENABLE_BASH_COMPLETION}
+ Enable fedabipkgdiff : ${ENABLE_FEDABIPKGDIFF}
Generate html apidoc : ${ENABLE_APIDOC}
Generate html manual : ${ENABLE_MANUAL}
])
diff --git a/tools/Makefile.am b/tools/Makefile.am
index 6125842..6ab5aae 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -6,6 +6,12 @@ else
bin_PROGRAMS = abidiff abilint abidw abicompat abipkgdiff
endif
+if ENABLE_FEDABIPKGDIFF
+ bin_SCRIPTS = fedabipkgdiff
+else
+ noinst_SCRIPTS = fedabipkgdiff
+endif
+
noinst_PROGRAMS = abisym abinilint
if ENABLE_ZIP_ARCHIVE
--
1.8.3.1
[-- Attachment #3: Type: text/plain, Size: 13 bytes --]
--
Dodji
^ permalink raw reply [flat|nested] 10+ messages in thread
* Fix adding fedabipkgdiff to source distribution tarball
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Dodji Seketeli
@ 2016-01-01 0:00 ` Dodji Seketeli
1 sibling, 0 replies; 10+ messages in thread
From: Dodji Seketeli @ 2016-01-01 0:00 UTC (permalink / raw)
To: Chenxiong Qi; +Cc: libabigail
Hello,
Dodji Seketeli <dodji@redhat.com> a écrit:
> I see that the patch is missing some changes necessary to integrate it
> to the autotool machinery and have the 'configure' script check for the
> dependencies of the fedabipkgdiff tool (mostly all the python modules
> and whatnot). I know this can look dreadful at first sight, so I made
> the changes. I have attached the patch at the end of this message.
In the autotool integration patch that I posted earlier, I forgot to
actually make the fedabipkgdiff python script be included in the source
distrubtion that is constructed when the user invokes the standard "make
dist" command.
This patch fixes it:
---------------------------->8<------------------------------
Add fedabipkgdiff script to source distribution
I forgot to add the fedabipkgdiff python script to source
distribution. The tree was thus not passing "make distcheck".
Fixed thus.
* tools/Makefile.am: Add fedabipkgdiff to source distribution.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
diff --git a/tools/Makefile.am b/tools/Makefile.am
index 6ab5aae..ba72567 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -14,6 +14,8 @@ endif
noinst_PROGRAMS = abisym abinilint
+EXTRA_DIST = fedabipkgdiff
+
if ENABLE_ZIP_ARCHIVE
abiar_SOURCES = abiar.cc
abiardir = $(bindir)
---------------------------->8<------------------------------
I have thus merged this patch into the initial autotool patch I made.
You can thus see it in my updated 'dodji/fedabipkgdiff' branch in the
upstream git repository at
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=11861ac897f5d72c0308e7188617755200994ff9.
Cheers,
--
Dodji
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 a new tool fedabipkgdiff Chenxiong Qi
2016-01-01 0:00 ` Chenxiong Qi
@ 2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
1 sibling, 1 reply; 10+ messages in thread
From: Chenxiong Qi @ 2016-01-01 0:00 UTC (permalink / raw)
To: libabigail, dodji
[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]
An updated patch with a small fix.
Regards,
Chenxiong Qi
----- Original Message -----
> From: "Chenxiong Qi" <cqi@redhat.com>
> To: libabigail@sourceware.org, dodji@redhat.com
> Sent: Wednesday, February 17, 2016 6:38:31 PM
> Subject: a new tool fedabipkgdiff
>
> Hi,
>
> This is a new tool fedabipkgdiff that would be much convenient for
> Fedora packagers to check potential ABI/API differences quickly using
> abipkgdiff shipped with libabigail. This tool came from a cool idea from
> Dodji. Currently, as the first step, it supports following ways,
>
> fedabipkgdiff --from fc23 ./foo-0.1-1.fc23.x86_64.rpm
> fedabipkgdiff --from fc23 --to fc24 foo
> fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
> fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
>
> For more details, please refer to
> https://sourceware.org/bugzilla/show_bug.cgi?id=19428
>
> Next step is to support the 4th use case mentioned in bug 19428.
>
> fedabipkgdiff is being under development, still need to improve. Welcome
> any feedback. Thanks.
>
> Happy hacking :)
>
> Regards,
> Chenxiong Qi
>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-new-tool-of-fedabipkgdiff.patch --]
[-- Type: text/x-patch; name=0001-new-tool-of-fedabipkgdiff.patch, Size: 31819 bytes --]
From 245ade840f7515d0319bb25c6c94c7171f7054f1 Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Tue, 9 Feb 2016 18:05:33 +0800
Subject: [PATCH] new tool of fedabipkgdiff
fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
compatibility issues quickly.
Currently with the first version of fedabipkgdiff, you can invoke it in
following ways.
fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
Bug 19428
---
.gitignore | 3 +
tests/Makefile.am | 4 +
tests/runtestfedabipkgdiff.sh | 5 +
tools/fedabipkgdiff | 855 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 867 insertions(+)
create mode 100755 tests/runtestfedabipkgdiff.sh
create mode 100755 tools/fedabipkgdiff
diff --git a/.gitignore b/.gitignore
index bb7c42a..169400a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ Makefile.in
*.lo
*.o
*~
+*.swp
/aclocal.m4
/autom4te.cache/
@@ -17,3 +18,5 @@ Makefile.in
/include/abg-version.h
/*.pc
+
+.tags
diff --git a/tests/Makefile.am b/tests/Makefile.am
index caf49e6..958995c 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,6 +31,7 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
+runtestfedabipkgdiff.sh \
$(CXX11_TESTS)
EXTRA_DIST = runtestcanonicalizetypes.sh.in
@@ -114,6 +115,9 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
+runtestfedabipkgdiff_sh_SOURCES =
+runtestfedabipkgdiff.sh$(EXEEXT):
+
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.sh b/tests/runtestfedabipkgdiff.sh
new file mode 100755
index 0000000..4144419
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.sh
@@ -0,0 +1,5 @@
+#!/usr/bin/bash
+
+export FEDABIPKGDIFF_TESTS=1
+
+../../tools/fedabipkgdiff
\ No newline at end of file
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
new file mode 100755
index 0000000..5d62db2
--- /dev/null
+++ b/tools/fedabipkgdiff
@@ -0,0 +1,855 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import argparse
+import glob
+import logging
+import os
+import re
+import shlex
+import subprocess
+import sys
+
+from itertools import groupby
+from urlparse import urlparse
+
+import koji
+
+"""
+Find proper RPM packages from koji to run abipkgdiff.
+
+Internal structure is
+
+fc23 fc24
+i686 i686
+ foo-0.1-1.fc23.i686.rpm foo-0.2-1.fc24.i686.rpm
+ foo-debuginfo-0.1-1.fc23.i686.rpm foo-debuginfo-0.2-1.fc23.i686.rpm
+x86_64 x86_64
+ foo-0.1-1.fc23.x86_64.rpm foo-0.2-1.fc24.x86_64.rpm
+ foo-debuginfo-0.1-1.fc23.x86_64.rpm foo-debuginfo-0.2-1.fc23.x86_64.rpm
+"""
+
+DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
+DEFAULT_KOJI_SERVER = 'http://koji.fedoraproject.org/kojihub'
+
+HOME_DIR = os.path.join('/tmp',
+ os.path.splitext(os.path.basename(__file__))[0])
+
+global_config = None
+
+logging.basicConfig(format='%(levelname)s %(asctime)s %(name)s %(message)s',
+ level=logging.DEBUG)
+logger = logging.getLogger(os.path.basename(__file__))
+
+
+class KojiPackageNotFound(Exception):
+ """Package is not found in Koji"""
+
+
+class PackageNotFound(Exception):
+ """Package is not found locally"""
+
+
+class InvalidFedoraDistro(Exception):
+ """Invalid Fedora Distro"""
+
+
+class CannotFindLatestBuildError(Exception):
+ """Cannot find latest build from a package"""
+
+
+def is_fedora_distro(distro):
+ """Adjust if a distro is specific to Fedora
+
+ :param str distro: a string representing a distro value.
+ :return: True if distro is the one specific to Fedora, like fc5, fc24.
+ "rtype: bool
+ """
+ return re.match(r'^fc\d{1,2}$', distro) is not None
+
+
+class Brew(object):
+ """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff"""
+
+ def __init__(self, baseurl):
+ self.session = koji.ClientSession(baseurl)
+
+ def listRPMs(self, **kwargs):
+ selector = kwargs.pop('selector')
+ rpms = self.session.listRPMs(**kwargs)
+
+ if selector:
+ rpms = [rpm for rpm in rpms if selector(rpm)]
+
+ return rpms
+
+ def getRPM(self, *args, **kwargs):
+ rpm = self.session.getRPM(*args, **kwargs)
+ if rpm is None:
+ raise KojipackageNotFound()
+ return rpm
+
+ def listBuilds(self, topone=None, selector=None, order_by=None,
+ reverse=None, **kwargs):
+ """Proxy to kojihub.listBuilds
+
+ Suport additional two keyword parameters:
+
+ - topone: return the top first one
+ - selector: a callable object used to select specific subset of builds
+ """
+ if 'state' not in kwargs:
+ kwargs['state'] = koji.BUILD_STATES['COMPLETE']
+
+ if selector is not None and not hasattr(selector, '__call__'):
+ raise TypeError(
+ '{0} is not a callable object.'.foramt(str(selector)))
+
+ if order_by is not None and not isinstance(order_by, basestring):
+ raise TypeError('order_by {0} is invalid.'.format(order_by))
+
+ builds = self.session.listBuilds(**kwargs)
+ if selector is not None:
+ builds = [build for build in builds if selector(build)]
+ if order_by is not None:
+ # FIXME: is it possible to sort builds by using opts parameter of
+ # listBuilds
+ builds = sorted(builds,
+ key=lambda item: item[order_by],
+ reverse=reverse)
+ if topone:
+ builds = builds[0:1]
+
+ return builds
+
+ def getPackage(self, name):
+ package = self.session.getPackage(name)
+ if package is None:
+ package = self.session.getPackage(name.rsplit('-', 1)[0])
+ if package is None:
+ raise KojiPackageNotFound()
+ return package
+
+ def getBuild(self, *args, **kwargs):
+ return self.session.getBuild(*args, **kwargs)
+
+ def get_rpm_build_id(self, name, version, release, arch=None):
+ """Get build ID that contains a rpm with specific nvra
+
+ If arch is omitted, a rpm can be identified easily by its N-V-R-A.
+
+ If arch is omitted, name is used to get associated package, and then
+ to get the build.
+
+ :param str name: name of a rpm
+ :param str version: version of a rpm
+ :param str release: release of a rpm
+ :param arch: arch of a rpm
+ :type arch: str or None
+ :return: the build from where the rpm is built
+ :rtype: dict
+ :raises KojiPackageNotFound: if name is not found from koji when arch
+ is None
+ """
+ if arch is None:
+ package = self.getPackage(name)
+ selector = lambda item: item['version'] == version and \
+ item['release'] == release
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector)
+ return builds[0]['build_id']
+ else:
+ rpm = self.getRPM({'name': name,
+ 'version': version,
+ 'release': release,
+ 'arch': arch,
+ })
+ return rpm['build_id']
+
+ def get_package_latest_build(self, package_name, distro):
+ """Get latest build from a package
+
+ :param str package_name: from which package to get the latest build
+ :param str distro: which distro the latest build belongs to
+ :return: the found build
+ :rtype: dict or None
+ """
+ package = self.getPackage(package_name)
+ selector = lambda item: item['release'].endswith(distro)
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector,
+ order_by='nvr',
+ reverse=True)
+ return builds[0] if builds else None
+
+ def select_rpms_from_a_build(self, build_id, package_name, arches=None):
+ """Select specific RPMs within a build
+
+ rpms could be filtered be specific criterias by the parameters.
+
+ :param int build_id: from which build to select rpms.
+ :param str package_name: which rpm to select that matches this name.
+ :param arches: which arches to select. If arches omits, rpms with all
+ arches except noarch and src will be selected.
+ :type arches: list, tuple or None
+ :return: a list of rpms returned from listRPMs
+ :rtype: list
+ """
+ def rpms_selector(package_name, excluded_arches):
+ return lambda rpm: \
+ rpm['arch'] not in excluded_arches and \
+ (rpm['name'] == package_name or
+ rpm['name'].endswith('-debuginfo'))
+
+ selector = rpms_selector(package_name, ('noarch', 'src'))
+ return self.listRPMs(buildID=build_id,
+ arches=arches,
+ selector=selector)
+
+ def get_latest_built_rpms(self, package_name, distro, arches=None):
+ """Get rpms from latest build of a package
+
+ By default, debuginfo rpm is also retrieved.
+
+ :param str package_name: from which package to get the rpms
+ :param str distro: which distro the rpms belong to
+ :param arches: which arches the rpms belong to
+ :type arches: str or None
+ :return: the selected rpms
+ :rtype: list
+ """
+ latest_build = self.get_package_latest_build(package_name, distro)
+
+ # Get rpm and debuginfo rpm from each arch
+ return self.select_rpms_from_a_build(latest_build['build_id'],
+ package_name,
+ arches=arches)
+
+
+def get_session():
+ return Brew(global_config.koji_server)
+
+
+def get_download_dir():
+ download_dir = os.path.join(HOME_DIR, 'downloads')
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
+
+def download_rpm(url):
+ cmd = shlex.split('wget -q -P {0} -c {1}'.format(get_download_dir(), url))
+ proc = subprocess.Popen(cmd)
+ s_stdout, s_stderr = proc.communicate()
+ if proc.returncode > 0:
+ logger.error('wget fails. returned code: %d. message: %s',
+ proc.returncode, s_stderr)
+ return False
+ return True
+
+
+def download_rpms(pkg_info):
+ def _download(pkg_info):
+ if os.path.exists(pkg_info['downloaded_rpm_file']):
+ logger.info('Reuse %s', pkg_info['downloaded_rpm_file'])
+ else:
+ logger.info('Download %s', pkg_info['download_url'])
+ download_rpm(pkg_info['download_url'])
+
+ for arch, rpm_infos in pkg_info.iteritems():
+ map(_download, rpm_infos)
+
+
+def find_rpm_filepath(rpm):
+ """Build RPM download URL"""
+ path_info = koji.PathInfo(topdir=global_config.koji_topdir)
+ session = get_session()
+ build = session.getBuild(rpm['build_id'])
+ return os.path.join(path_info.build(build), path_info.rpm(rpm))
+
+
+def find_local_debuginfo_rpm(rpm_file):
+ """Find debuginfo rpm package from a directory
+
+ :param str rpm_file: the rpm file name
+ :return: the absolute file name of the found debuginfo rpm
+ :rtype: str or None
+ """
+ search_dir = os.path.dirname(os.path.abspath(rpm_file))
+ nvra = koji.parse_NVRA(os.path.basename(rpm_file))
+ debuginfo_rpm_file_glob = os.path.join(
+ search_dir,
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
+
+ try:
+ debuginfo_rpm = glob.glob(debuginfo_rpm_file_glob)[0]
+ except IndexError:
+ return None
+
+ return debuginfo_rpm
+
+
+def abipkgdiff(pkg1_info, pkg2_info):
+ """Run abipkgdiff against found two RPM packages"""
+
+ pkg1_rpm, pkg1_debuginfo_rpm = pkg1_info
+ pkg2_rpm, pkg2_debuginfo_rpm = pkg2_info
+
+ cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ pkg1_debuginfo_rpm, pkg2_debuginfo_rpm,
+ pkg1_rpm, pkg2_rpm)
+
+ if global_config.dry_run:
+ logger.info('DRY-RUN: %s', cmd)
+ return
+
+ logger.debug('Run: %s', cmd)
+
+ proc = subprocess.Popen(shlex.split(cmd))
+ return proc.wait()
+
+
+def run_abipkgdiff(pkg1_infos, pkg2_infos):
+ """Run abipkgdiff
+
+ If one of the executions finds ABI differences, the return code is the
+ return code from abipkgdiff.
+ """
+ arches = pkg1_infos.keys()
+ arches.sort()
+
+ return_code = 0
+
+ for arch in arches:
+ pkg1_info = pkg1_infos[arch]
+ pkg2_info = pkg2_infos[arch]
+
+ ret = abipkgdiff((pkg1_info[0]['downloaded_rpm_file'],
+ pkg1_info[1]['downloaded_rpm_file']),
+ (pkg2_info[0]['downloaded_rpm_file'],
+ pkg2_info[1]['downloaded_rpm_file']))
+ if ret > 0:
+ return_code = ret
+
+ return return_code
+
+
+def diff_local_rpm_with_latest_rpm_from_koji():
+ """Diff against local rpm and remove latest rpm
+
+ This operation handles a local rpm and debuginfo rpm and remote ones
+ located in remote Koji server, that has specific distro specificed by
+ argument --from.
+
+ 1/ Suppose the packager has just locally built a package named
+ foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
+ latest stable package from Fedora 23, one would do:
+
+ fedabidiff --from f23 ./foo-3.0.fc24.rpm
+ """
+
+ if not is_fedora_distro(global_config.from_distro):
+ raise InvalidFedoraDistro('Invalid Fedora distro {0}'.format(distro))
+
+ local_rpm_file = global_config.NVR[0]
+ if not os.path.exists(local_rpm_file):
+ raise ValueError('{0} does not exist.'.format(local_rpm_file))
+
+ local_debuginfo_rpm = find_local_debuginfo_rpm(local_rpm_file)
+ logger.debug('Found local debuginfo rpm %s', local_debuginfo_rpm)
+ if local_debuginfo_rpm is None:
+ raise ValueError(
+ 'debuginfo rpm {0} does not exist.'.format(local_debuginfo_rpm))
+
+ nvra = koji.parse_NVRA(os.path.basename(local_rpm_file))
+ session = get_session()
+ rpms = session.get_latest_built_rpms(nvra['name'],
+ global_config.from_distro,
+ arches=nvra['arch'])
+ pkg_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg_infos)
+
+ pkg_info = pkg_infos.values()[0]
+ return abipkgdiff((local_rpm_file, local_debuginfo_rpm),
+ (pkg_info[0]['downloaded_rpm_file'],
+ pkg_info[1]['downloaded_rpm_file']))
+
+
+def make_rpms_usable_for_abipkgdiff(rpms):
+ """
+ Construct result that contains mappings from arch to download url and
+ downloaded rpm filename of rpm and debuginfo rpm
+
+ :return: a mapping from an arch to a list of dict objects that contains
+ a URL from where to download the rpm, and an absolute path of a
+ predictable downloaded rpm filename.
+ :rtype: dict
+ """
+
+ result = {}
+
+ rpms_iter = groupby(sorted(rpms, key=lambda rpm: rpm['arch']),
+ key=lambda item: item['arch'])
+
+ for arch, rpms in rpms_iter:
+ l = []
+ # sorted ensures the order of rpm and associated debuginfo rpm
+ for item in sorted(rpms, key=lambda item: item['name']):
+ download_url = find_rpm_filepath(item)
+ l.append({
+ 'download_url': download_url,
+ 'downloaded_rpm_file': os.path.join(
+ get_download_dir(),
+ os.path.basename(urlparse(download_url).path)),
+ })
+ result[arch] = l
+
+ return result
+
+
+def diff_latest_rpms_based_on_distros():
+ """abipkgdiff rpms based on two distros
+
+ 2/ Suppose the packager wants to see how the ABIs of the package foo
+ evolved between fedora 19 and fedora 22. She would thus type the command:
+
+ fedabidiff --from f19 --to f22 foo
+ """
+
+ from_distro = global_config.from_distro
+ to_distro = global_config.to_distro
+
+ if not is_fedora_distro(from_distro):
+ raise InvalidFedoraDistro(
+ 'Invalid Fedora distro {0}'.format(from_distro))
+
+ if not is_fedora_distro(to_distro):
+ raise InvalidFedoraDistro(
+ 'Invalid Fedora distro {0}'.format(distro))
+
+ package_name = global_config.NVR[0]
+
+ session = get_session()
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.from_distro)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg1_infos)
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.to_distro)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def diff_rpms_with_nvra(name, version, release, arch=None):
+ session = get_session()
+
+ build_id = session.get_rpm_build_id(name, version, release, arch)
+ rpms = session.select_rpms_from_a_build(build_id, name, arches=arch)
+ return make_rpms_usable_for_abipkgdiff(rpms)
+
+
+def diff_two_nvras_from_koji():
+ """Diff two nvras from koji
+
+ The arch probably omits, that means febabipkgdiff will diff all arches. If
+ specificed, the specific arch will be handled.
+
+ 3/ Suppose the packager wants to compare the ABI of two packages designated
+ by their name and version. She would issue a command like this:
+
+ fedabidiff foo-1.0.fc19 foo-3.0.fc24
+ fedabidiff foo-1.0.fc19.i686 foo-1.0.fc24.i686
+ """
+ left_rpm = koji.parse_NVRA(global_config.NVR[0])
+ right_rpm = koji.parse_NVRA(global_config.NVR[1])
+
+ if is_fedora_distro(left_rpm['arch']) and \
+ is_fedora_distro(right_rpm['arch']):
+ nvr = koji.parse_NVR(global_config.NVR[0])
+ params1 = (nvr['name'], nvr['version'], nvr['release'])
+
+ nvr = koji.parse_NVR(global_config.NVR[1])
+ params2 = (nvr['name'], nvr['version'], nvr['release'])
+ else:
+ params1 = (left_rpm['name'],
+ left_rpm['version'],
+ left_rpm['release'],
+ left_rpm['arch'])
+ params2 = (right_rpm['name'],
+ right_rpm['version'],
+ right_rpm['release'],
+ right_rpm['arch'])
+
+ pkg1_infos = diff_rpms_with_nvra(*params1)
+ download_rpms(pkg1_infos)
+
+ pkg2_infos = diff_rpms_with_nvra(*params2)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def build_commandline_args_parser():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('NVR', nargs='+')
+ parser.add_argument('--server', required=False, dest='koji_server',
+ default=DEFAULT_KOJI_SERVER)
+ parser.add_argument('--topdir', required=False, dest='koji_topdir',
+ default=DEFAULT_KOJI_TOPDIR)
+ parser.add_argument('--dry-run', required=False, dest='dry_run',
+ action='store_true')
+ parser.add_argument('--from', required=False, metavar='DISTRO',
+ dest='from_distro')
+ parser.add_argument('--to', required=False, metavar='DISTRO',
+ dest='to_distro')
+
+ return parser
+
+
+def main():
+ parser = build_commandline_args_parser()
+
+ args = parser.parse_args()
+
+ global global_config
+ global_config = args
+
+ if global_config.from_distro and global_config.to_distro is None and \
+ global_config.NVR:
+ returncode = diff_local_rpm_with_latest_rpm_from_koji()
+
+ elif global_config.from_distro and global_config.to_distro and \
+ global_config.NVR:
+ returncode = diff_latest_rpms_based_on_distros()
+
+ elif global_config.from_distro is None and \
+ global_config.to_distro is None and len(global_config.NVR) > 1:
+ returncode = diff_two_nvras_from_koji()
+
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to -h.'
+ returncode = 1
+
+ return returncode
+
+
+invoked_from_cmd = __name__ == '__main__'
+
+if 'FEDABIPKGDIFF_TESTS' not in os.environ and invoked_from_cmd:
+ try:
+ sys.exit(main())
+ except Exception as e:
+ print >>sys.stderr, str(e)
+ sys.exit(1)
+
+
+import itertools
+import shutil
+import unittest
+
+
+try:
+ from mock import patch
+except ImportError:
+ print >>sys.stderr, \
+ 'mock is not installed. Please install it before running tests.'
+ sys.exit(1)
+
+counter = itertools.count(0)
+
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_fedora_distro(self):
+ distro = 'fc5'
+ self.assertTrue(is_fedora_distro(distro))
+
+ distro = 'f5'
+ self.assertFalse(is_fedora_distro(distro))
+
+ distro = 'fc23'
+ self.assertTrue(is_fedora_distro(distro))
+
+ distro = 'fc'
+ self.assertFalse(is_fedora_distro(distro))
+
+ distro = 'fc234'
+ self.assertFalse(is_fedora_distro(distro))
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff"""
+
+ def setUp(self):
+ self.pkg1_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg1_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ @patch('__main__.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 0
+
+ result = run_abipkgdiff(self.pkg1_single_info, self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('__main__.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 4
+
+ result = run_abipkgdiff(self.pkg1_single_info, self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('__main__.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+fake_rpm_file = 'foo-0.1-1.fc24.x86_64.rpm'
+fake_debuginfo_rpm_file = 'foo-debuginfo-0.1-1.fc24.x86_64.rpm'
+
+
+class MockGlobalConfig(object):
+ koji_server = DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ return MockKojiClientSession(baseurl=DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock"""
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class SelectRpmsFromABuildTest(unittest.TestCase):
+ """Test case for select_rpms_from_a_build"""
+
+ def assert_rpms(self, rpms):
+ for item in rpms:
+ self.assertTrue(item['arch'] in ['i686', 'x86_64'])
+ self.assertTrue(item['name'] in ('httpd', 'httpd-debuginfo'))
+
+ @patch('__main__.Brew.listRPMs')
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_all_arches(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+ @patch('__main__.Brew.listRPMs')
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_one_arch(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ session = get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_fail_to_find_latest_build(self):
+ session = get_session()
+ latest_build = session.get_package_latest_build('httpd', 'xxxx')
+ self.assertEquals(None, latest_build)
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ session = get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+class FindLocalDebuginfoRPMTest(unittest.TestCase):
+ """Test case for find_local_debuginfo_rpm"""
+
+ def setUp(self):
+ # FIXME: is it possible to patch glob or the underlying methods glob
+ # depends on
+ self.test_dir = './test_dir'
+ os.makedirs(self.test_dir)
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir, fake_debuginfo_rpm_file)))
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file.replace('foo', 'another'))))
+
+ def tearDown(self):
+ shutil.rmtree(self.test_dir)
+
+ def test_find_debuginfo_rpm(self):
+ debuginfo_rpm = find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, fake_rpm_file))
+
+ expected_debuginfo_rpm_file = os.path.abspath(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file))
+ self.assertEquals(expected_debuginfo_rpm_file, debuginfo_rpm)
+
+ def test_no_suitable_debuginfo_rpm(self):
+ debuginfo_rpm = find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, 'abc-0.1-1.i686.rpm'))
+ self.assertEquals(None, debuginfo_rpm)
+
+
+if invoked_from_cmd:
+ unittest.main()
--
2.5.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* a new tool fedabipkgdiff
@ 2016-01-01 0:00 Chenxiong Qi
2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Chenxiong Qi
0 siblings, 2 replies; 10+ messages in thread
From: Chenxiong Qi @ 2016-01-01 0:00 UTC (permalink / raw)
To: libabigail, dodji
[-- Attachment #1: Type: text/plain, Size: 770 bytes --]
Hi,
This is a new tool fedabipkgdiff that would be much convenient for
Fedora packagers to check potential ABI/API differences quickly using
abipkgdiff shipped with libabigail. This tool came from a cool idea from
Dodji. Currently, as the first step, it supports following ways,
fedabipkgdiff --from fc23 ./foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
For more details, please refer to
https://sourceware.org/bugzilla/show_bug.cgi?id=19428
Next step is to support the 4th use case mentioned in bug 19428.
fedabipkgdiff is being under development, still need to improve. Welcome
any feedback. Thanks.
Happy hacking :)
Regards,
Chenxiong Qi
[-- Attachment #2: 0001-new-tool-of-fedabipkgdiff.patch --]
[-- Type: text/x-patch, Size: 31789 bytes --]
From 7a73c612d253ff5600324b8fbc0604d2aa7be95b Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Tue, 9 Feb 2016 18:05:33 +0800
Subject: [PATCH] new tool of fedabipkgdiff
fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
compatibility issues quickly.
Currently with the first version of fedabipkgdiff, you can invoke it in
following ways.
fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
Bug 19428
---
.gitignore | 3 +
tests/Makefile.am | 4 +
tests/runtestfedabipkgdiff.sh | 5 +
tools/fedabipkgdiff | 854 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 866 insertions(+)
create mode 100755 tests/runtestfedabipkgdiff.sh
create mode 100755 tools/fedabipkgdiff
diff --git a/.gitignore b/.gitignore
index bb7c42a..169400a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ Makefile.in
*.lo
*.o
*~
+*.swp
/aclocal.m4
/autom4te.cache/
@@ -17,3 +18,5 @@ Makefile.in
/include/abg-version.h
/*.pc
+
+.tags
diff --git a/tests/Makefile.am b/tests/Makefile.am
index caf49e6..958995c 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,6 +31,7 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
+runtestfedabipkgdiff.sh \
$(CXX11_TESTS)
EXTRA_DIST = runtestcanonicalizetypes.sh.in
@@ -114,6 +115,9 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
+runtestfedabipkgdiff_sh_SOURCES =
+runtestfedabipkgdiff.sh$(EXEEXT):
+
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.sh b/tests/runtestfedabipkgdiff.sh
new file mode 100755
index 0000000..4144419
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.sh
@@ -0,0 +1,5 @@
+#!/usr/bin/bash
+
+export FEDABIPKGDIFF_TESTS=1
+
+../../tools/fedabipkgdiff
\ No newline at end of file
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
new file mode 100755
index 0000000..3e0096b
--- /dev/null
+++ b/tools/fedabipkgdiff
@@ -0,0 +1,854 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+import argparse
+import glob
+import logging
+import os
+import re
+import shlex
+import subprocess
+import sys
+
+from urlparse import urlparse
+
+import koji
+
+"""
+Find proper RPM packages from koji to run abipkgdiff.
+
+Internal structure is
+
+fc23 fc24
+i686 i686
+ foo-0.1-1.fc23.i686.rpm foo-0.2-1.fc24.i686.rpm
+ foo-debuginfo-0.1-1.fc23.i686.rpm foo-debuginfo-0.2-1.fc23.i686.rpm
+x86_64 x86_64
+ foo-0.1-1.fc23.x86_64.rpm foo-0.2-1.fc24.x86_64.rpm
+ foo-debuginfo-0.1-1.fc23.x86_64.rpm foo-debuginfo-0.2-1.fc23.x86_64.rpm
+"""
+
+DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
+DEFAULT_KOJI_SERVER = 'http://koji.fedoraproject.org/kojihub'
+
+HOME_DIR = os.path.join('/tmp',
+ os.path.splitext(os.path.basename(__file__))[0])
+
+global_config = None
+
+logging.basicConfig(format='%(levelname)s %(asctime)s %(name)s %(message)s',
+ level=logging.DEBUG)
+logger = logging.getLogger(os.path.basename(__file__))
+
+
+class KojiPackageNotFound(Exception):
+ """Package is not found in Koji"""
+
+
+class PackageNotFound(Exception):
+ """Package is not found locally"""
+
+
+class InvalidFedoraDistro(Exception):
+ """Invalid Fedora Distro"""
+
+
+class CannotFindLatestBuildError(Exception):
+ """Cannot find latest build from a package"""
+
+
+def is_fedora_distro(distro):
+ """Adjust if a distro is specific to Fedora
+
+ :param str distro: a string representing a distro value.
+ :return: True if distro is the one specific to Fedora, like fc5, fc24.
+ "rtype: bool
+ """
+ return re.match(r'^fc\d{1,2}$', distro) is not None
+
+
+class Brew(object):
+ """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff"""
+
+ def __init__(self, baseurl):
+ self.session = koji.ClientSession(baseurl)
+
+ def listRPMs(self, **kwargs):
+ selector = kwargs.pop('selector')
+ rpms = self.session.listRPMs(**kwargs)
+
+ if selector:
+ rpms = [rpm for rpm in rpms if selector(rpm)]
+
+ return rpms
+
+ def getRPM(self, *args, **kwargs):
+ rpm = self.session.getRPM(*args, **kwargs)
+ if rpm is None:
+ raise KojipackageNotFound()
+ return rpm
+
+ def listBuilds(self, topone=None, selector=None, order_by=None,
+ reverse=None, **kwargs):
+ """Proxy to kojihub.listBuilds
+
+ Suport additional two keyword parameters:
+
+ - topone: return the top first one
+ - selector: a callable object used to select specific subset of builds
+ """
+ if 'state' not in kwargs:
+ kwargs['state'] = koji.BUILD_STATES['COMPLETE']
+
+ if selector is not None and not hasattr(selector, '__call__'):
+ raise TypeError(
+ '{0} is not a callable object.'.foramt(str(selector)))
+
+ if order_by is not None and not isinstance(order_by, basestring):
+ raise TypeError('order_by {0} is invalid.'.format(order_by))
+
+ builds = self.session.listBuilds(**kwargs)
+ if selector is not None:
+ builds = [build for build in builds if selector(build)]
+ if order_by is not None:
+ # FIXME: is it possible to sort builds by using opts parameter of
+ # listBuilds
+ builds = sorted(builds,
+ key=lambda item: item[order_by],
+ reverse=reverse)
+ if topone:
+ builds = builds[0:1]
+
+ return builds
+
+ def getPackage(self, name):
+ package = self.session.getPackage(name)
+ if package is None:
+ package = self.session.getPackage(name.rsplit('-', 1)[0])
+ if package is None:
+ raise KojiPackageNotFound()
+ return package
+
+ def getBuild(self, *args, **kwargs):
+ return self.session.getBuild(*args, **kwargs)
+
+ def get_rpm_build_id(self, name, version, release, arch=None):
+ """Get build ID that contains a rpm with specific nvra
+
+ If arch is omitted, a rpm can be identified easily by its N-V-R-A.
+
+ If arch is omitted, name is used to get associated package, and then
+ to get the build.
+
+ :param str name: name of a rpm
+ :param str version: version of a rpm
+ :param str release: release of a rpm
+ :param arch: arch of a rpm
+ :type arch: str or None
+ :return: the build from where the rpm is built
+ :rtype: dict
+ :raises KojiPackageNotFound: if name is not found from koji when arch
+ is None
+ """
+ if arch is None:
+ package = self.getPackage(name)
+ selector = lambda item: item['version'] == version and \
+ item['release'] == release
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector)
+ return builds[0]['build_id']
+ else:
+ rpm = self.getRPM({'name': name,
+ 'version': version,
+ 'release': release,
+ 'arch': arch,
+ })
+ return rpm['build_id']
+
+ def get_package_latest_build(self, package_name, distro):
+ """Get latest build from a package
+
+ :param str package_name: from which package to get the latest build
+ :param str distro: which distro the latest build belongs to
+ :return: the found build
+ :rtype: dict or None
+ """
+ package = self.getPackage(package_name)
+ selector = lambda item: item['release'].endswith(distro)
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector,
+ order_by='nvr',
+ reverse=True)
+ return builds[0] if builds else None
+
+ def select_rpms_from_a_build(self, build_id, package_name, arches=None):
+ """Select specific RPMs within a build
+
+ rpms could be filtered be specific criterias by the parameters.
+
+ :param int build_id: from which build to select rpms.
+ :param str package_name: which rpm to select that matches this name.
+ :param arches: which arches to select. If arches omits, rpms with all
+ arches except noarch and src will be selected.
+ :type arches: list, tuple or None
+ :return: a list of rpms returned from listRPMs
+ :rtype: list
+ """
+ def rpms_selector(package_name, excluded_arches):
+ return lambda rpm: \
+ rpm['arch'] not in excluded_arches and \
+ (rpm['name'] == package_name or
+ rpm['name'].endswith('-debuginfo'))
+
+ selector = rpms_selector(package_name, ('noarch', 'src'))
+ return self.listRPMs(buildID=build_id,
+ arches=arches,
+ selector=selector)
+
+ def get_latest_built_rpms(self, package_name, distro, arches=None):
+ """Get rpms from latest build of a package
+
+ By default, debuginfo rpm is also retrieved.
+
+ :param str package_name: from which package to get the rpms
+ :param str distro: which distro the rpms belong to
+ :param arches: which arches the rpms belong to
+ :type arches: str or None
+ :return: the selected rpms
+ :rtype: list
+ """
+ latest_build = self.get_package_latest_build(package_name, distro)
+
+ # Get rpm and debuginfo rpm from each arch
+ return self.select_rpms_from_a_build(latest_build['build_id'],
+ package_name,
+ arches=arches)
+
+
+def get_session():
+ return Brew(global_config.koji_server)
+
+
+def get_download_dir():
+ download_dir = os.path.join(HOME_DIR, 'downloads')
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
+
+def download_rpm(url):
+ cmd = shlex.split('wget -q -P {0} -c {1}'.format(get_download_dir(), url))
+ proc = subprocess.Popen(cmd)
+ s_stdout, s_stderr = proc.communicate()
+ if proc.returncode > 0:
+ logger.error('wget fails. returned code: %d. message: %s',
+ proc.returncode, s_stderr)
+ return False
+ return True
+
+
+def download_rpms(pkg_info):
+ def _download(pkg_info):
+ if os.path.exists(pkg_info['downloaded_rpm_file']):
+ logger.info('Reuse %s', pkg_info['downloaded_rpm_file'])
+ else:
+ logger.info('Download %s', pkg_info['download_url'])
+ download_rpm(pkg_info['download_url'])
+
+ for arch, rpm_infos in pkg_info.iteritems():
+ map(_download, rpm_infos)
+
+
+def find_rpm_filepath(rpm):
+ """Build RPM download URL"""
+ path_info = koji.PathInfo(topdir=global_config.koji_topdir)
+ session = get_session()
+ build = session.getBuild(rpm['build_id'])
+ return os.path.join(path_info.build(build), path_info.rpm(rpm))
+
+
+def find_local_debuginfo_rpm(rpm_file):
+ """Find debuginfo rpm package from a directory
+
+ :param str rpm_file: the rpm file name
+ :return: the absolute file name of the found debuginfo rpm
+ :rtype: str or None
+ """
+ search_dir = os.path.dirname(os.path.abspath(rpm_file))
+ nvra = koji.parse_NVRA(os.path.basename(rpm_file))
+ debuginfo_rpm_file_glob = os.path.join(
+ search_dir,
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
+
+ try:
+ debuginfo_rpm = glob.glob(debuginfo_rpm_file_glob)[0]
+ except IndexError:
+ return None
+
+ return debuginfo_rpm
+
+
+def abipkgdiff(pkg1_info, pkg2_info):
+ """Run abipkgdiff against found two RPM packages"""
+
+ pkg1_rpm, pkg1_debuginfo_rpm = pkg1_info
+ pkg2_rpm, pkg2_debuginfo_rpm = pkg2_info
+
+ cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ pkg1_debuginfo_rpm, pkg2_debuginfo_rpm,
+ pkg1_rpm, pkg2_rpm)
+
+ if global_config.dry_run:
+ logger.info('DRY-RUN: %s', cmd)
+ return
+
+ logger.debug('Run: %s', cmd)
+
+ proc = subprocess.Popen(shlex.split(cmd))
+ return proc.wait()
+
+
+def run_abipkgdiff(pkg1_infos, pkg2_infos):
+ """Run abipkgdiff
+
+ If one of the executions finds ABI differences, the return code is the
+ return code from abipkgdiff.
+ """
+ arches = pkg1_infos.keys()
+ arches.sort()
+
+ return_code = 0
+
+ for arch in arches:
+ pkg1_info = pkg1_infos[arch]
+ pkg2_info = pkg2_infos[arch]
+
+ ret = abipkgdiff((pkg1_info[0]['downloaded_rpm_file'],
+ pkg1_info[1]['downloaded_rpm_file']),
+ (pkg2_info[0]['downloaded_rpm_file'],
+ pkg2_info[1]['downloaded_rpm_file']))
+ if ret > 0:
+ return_code = ret
+
+ return return_code
+
+
+def diff_local_rpm_with_latest_rpm_from_koji():
+ """Diff against local rpm and remove latest rpm
+
+ This operation handles a local rpm and debuginfo rpm and remote ones
+ located in remote Koji server, that has specific distro specificed by
+ argument --from.
+
+ 1/ Suppose the packager has just locally built a package named
+ foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
+ latest stable package from Fedora 23, one would do:
+
+ fedabidiff --from f23 ./foo-3.0.fc24.rpm
+ """
+
+ if not is_fedora_distro(global_config.from_distro):
+ raise InvalidFedoraDistro('Invalid Fedora distro {0}'.format(distro))
+
+ local_rpm_file = global_config.NVR[0]
+ if not os.path.exists(local_rpm_file):
+ raise ValueError('{0} does not exist.'.format(local_rpm_file))
+
+ local_debuginfo_rpm = find_local_debuginfo_rpm(local_rpm_file)
+ logger.debug('Found local debuginfo rpm %s', local_debuginfo_rpm)
+ if local_debuginfo_rpm is None:
+ raise ValueError(
+ 'debuginfo rpm {0} does not exist.'.format(local_debuginfo_rpm))
+
+ nvra = koji.parse_NVRA(os.path.basename(local_rpm_file))
+ session = get_session()
+ rpms = session.get_latest_built_rpms(nvra['name'],
+ global_config.from_distro,
+ arches=nvra['arch'])
+ pkg_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg_infos)
+
+ pkg_info = pkg_infos.values()[0]
+ return abipkgdiff((local_rpm_file, local_debuginfo_rpm),
+ (pkg_info[0]['downloaded_rpm_file'],
+ pkg_info[1]['downloaded_rpm_file']))
+
+
+def make_rpms_usable_for_abipkgdiff(rpms):
+ """
+ Construct result that contains mappings from arch to download url and
+ downloaded rpm filename of rpm and debuginfo rpm
+
+ :return: a mapping from an arch to a list of dict objects that contains
+ a URL from where to download the rpm, and an absolute path of a
+ predictable downloaded rpm filename.
+ :rtype: dict
+ """
+
+ result = {}
+
+ rpms_iter = groupby(sorted(rpms, key=lambda rpm: rpm['arch']),
+ key=lambda item: item['arch'])
+
+ for arch, rpms in rpms_iter:
+ l = []
+ # sorted ensures the order of rpm and associated debuginfo rpm
+ for item in sorted(rpms, key=lambda item: item['name']):
+ download_url = find_rpm_filepath(item)
+ l.append({
+ 'download_url': download_url,
+ 'downloaded_rpm_file': os.path.join(
+ get_download_dir(),
+ os.path.basename(urlparse(download_url).path)),
+ })
+ result[arch] = l
+
+ return result
+
+
+def diff_latest_rpms_based_on_distros():
+ """abipkgdiff rpms based on two distros
+
+ 2/ Suppose the packager wants to see how the ABIs of the package foo
+ evolved between fedora 19 and fedora 22. She would thus type the command:
+
+ fedabidiff --from f19 --to f22 foo
+ """
+
+ from_distro = global_config.from_distro
+ to_distro = global_config.to_distro
+
+ if not is_fedora_distro(from_distro):
+ raise InvalidFedoraDistro(
+ 'Invalid Fedora distro {0}'.format(from_distro))
+
+ if not is_fedora_distro(to_distro):
+ raise InvalidFedoraDistro(
+ 'Invalid Fedora distro {0}'.format(distro))
+
+ package_name = global_config.NVR[0]
+
+ session = get_session()
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.from_distro)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg1_infos)
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.to_distro)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def diff_rpms_with_nvra(name, version, release, arch=None):
+ session = get_session()
+
+ build_id = session.get_rpm_build_id(name, version, release, arch)
+ rpms = session.select_rpms_from_a_build(build_id, name, arches=arch)
+ return make_rpms_usable_for_abipkgdiff(rpms)
+
+
+def diff_two_nvras_from_koji():
+ """Diff two nvras from koji
+
+ The arch probably omits, that means febabipkgdiff will diff all arches. If
+ specificed, the specific arch will be handled.
+
+ 3/ Suppose the packager wants to compare the ABI of two packages designated
+ by their name and version. She would issue a command like this:
+
+ fedabidiff foo-1.0.fc19 foo-3.0.fc24
+ fedabidiff foo-1.0.fc19.i686 foo-1.0.fc24.i686
+ """
+ left_rpm = koji.parse_NVRA(global_config.NVR[0])
+ right_rpm = koji.parse_NVRA(global_config.NVR[1])
+
+ if is_fedora_distro(left_rpm['arch']) and \
+ is_fedora_distro(right_rpm['arch']):
+ nvr = koji.parse_NVR(global_config.NVR[0])
+ params1 = (nvr['name'], nvr['version'], nvr['release'])
+
+ nvr = koji.parse_NVR(global_config.NVR[1])
+ params2 = (nvr['name'], nvr['version'], nvr['release'])
+ else:
+ params1 = (left_rpm['name'],
+ left_rpm['version'],
+ left_rpm['release'],
+ left_rpm['arch'])
+ params2 = (right_rpm['name'],
+ right_rpm['version'],
+ right_rpm['release'],
+ right_rpm['arch'])
+
+ pkg1_infos = diff_rpms_with_nvra(*params1)
+ download_rpms(pkg1_infos)
+
+ pkg2_infos = diff_rpms_with_nvra(*params2)
+ download_rpms(pkg2_infos)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def build_commandline_args_parser():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('NVR', nargs='+')
+ parser.add_argument('--server', required=False, dest='koji_server',
+ default=DEFAULT_KOJI_SERVER)
+ parser.add_argument('--topdir', required=False, dest='koji_topdir',
+ default=DEFAULT_KOJI_TOPDIR)
+ parser.add_argument('--dry-run', required=False, dest='dry_run',
+ action='store_true')
+ parser.add_argument('--from', required=False, metavar='DISTRO',
+ dest='from_distro')
+ parser.add_argument('--to', required=False, metavar='DISTRO',
+ dest='to_distro')
+
+ return parser
+
+
+def main():
+ parser = build_commandline_args_parser()
+
+ args = parser.parse_args()
+
+ global global_config
+ global_config = args
+
+ if global_config.from_distro and global_config.to_distro is None and \
+ global_config.NVR:
+ returncode = diff_local_rpm_with_latest_rpm_from_koji()
+
+ elif global_config.from_distro and global_config.to_distro and \
+ global_config.NVR:
+ returncode = diff_latest_rpms_based_on_distros()
+
+ elif global_config.from_distro is None and \
+ global_config.to_distro is None and len(global_config.NVR) > 1:
+ returncode = diff_two_nvras_from_koji()
+
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to -h.'
+ returncode = 1
+
+ return returncode
+
+
+invoked_from_cmd = __name__ == '__main__'
+
+if 'FEDABIPKGDIFF_TESTS' not in os.environ and invoked_from_cmd:
+ try:
+ sys.exit(main())
+ except Exception as e:
+ print >>sys.stderr, str(e)
+ sys.exit(1)
+
+
+import itertools
+import shutil
+import unittest
+
+
+try:
+ from mock import patch
+except ImportError:
+ print >>sys.stderr, \
+ 'mock is not installed. Please install it before running tests.'
+ sys.exit(1)
+
+counter = itertools.count(0)
+
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_fedora_distro(self):
+ distro = 'fc5'
+ self.assertTrue(is_fedora_distro(distro))
+
+ distro = 'f5'
+ self.assertFalse(is_fedora_distro(distro))
+
+ distro = 'fc23'
+ self.assertTrue(is_fedora_distro(distro))
+
+ distro = 'fc'
+ self.assertFalse(is_fedora_distro(distro))
+
+ distro = 'fc234'
+ self.assertFalse(is_fedora_distro(distro))
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff"""
+
+ def setUp(self):
+ self.pkg1_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg1_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ @patch('__main__.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 0
+
+ result = run_abipkgdiff(self.pkg1_single_info, self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('__main__.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 4
+
+ result = run_abipkgdiff(self.pkg1_single_info, self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('__main__.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ result = run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+fake_rpm_file = 'foo-0.1-1.fc24.x86_64.rpm'
+fake_debuginfo_rpm_file = 'foo-debuginfo-0.1-1.fc24.x86_64.rpm'
+
+
+class MockGlobalConfig(object):
+ koji_server = DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ return MockKojiClientSession(baseurl=DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock"""
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class SelectRpmsFromABuildTest(unittest.TestCase):
+ """Test case for select_rpms_from_a_build"""
+
+ def assert_rpms(self, rpms):
+ for item in rpms:
+ self.assertTrue(item['arch'] in ['i686', 'x86_64'])
+ self.assertTrue(item['name'] in ('httpd', 'httpd-debuginfo'))
+
+ @patch('__main__.Brew.listRPMs')
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_all_arches(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+ @patch('__main__.Brew.listRPMs')
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_one_arch(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ session = get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_fail_to_find_latest_build(self):
+ session = get_session()
+ latest_build = session.get_package_latest_build('httpd', 'xxxx')
+ self.assertEquals(None, latest_build)
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('__main__.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ session = get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+class FindLocalDebuginfoRPMTest(unittest.TestCase):
+ """Test case for find_local_debuginfo_rpm"""
+
+ def setUp(self):
+ # FIXME: is it possible to patch glob or the underlying methods glob
+ # depends on
+ self.test_dir = './test_dir'
+ os.makedirs(self.test_dir)
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir, fake_debuginfo_rpm_file)))
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file.replace('foo', 'another'))))
+
+ def tearDown(self):
+ shutil.rmtree(self.test_dir)
+
+ def test_find_debuginfo_rpm(self):
+ debuginfo_rpm = find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, fake_rpm_file))
+
+ expected_debuginfo_rpm_file = os.path.abspath(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file))
+ self.assertEquals(expected_debuginfo_rpm_file, debuginfo_rpm)
+
+ def test_no_suitable_debuginfo_rpm(self):
+ debuginfo_rpm = find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, 'abc-0.1-1.i686.rpm'))
+ self.assertEquals(None, debuginfo_rpm)
+
+
+if invoked_from_cmd:
+ unittest.main()
--
2.5.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Dodji Seketeli
@ 2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Fix adding fedabipkgdiff to source distribution tarball Dodji Seketeli
1 sibling, 0 replies; 10+ messages in thread
From: Dodji Seketeli @ 2016-01-01 0:00 UTC (permalink / raw)
To: Chenxiong Qi; +Cc: libabigail
[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]
Hello,
Dodji Seketeli <dodji@redhat.com> a écrit:
[...]
>> --- /dev/null
>> +++ b/tests/runtestfedabipkgdiff.sh
[...]
>> @@ -0,0 +1,5 @@
>> +#!/usr/bin/bash
>> +
>> +export FEDABIPKGDIFF_TESTS=1
>> +
>> +../../tools/fedabipkgdiff
>
> Actually, I think the runtestfedabipkgdiff.sh could be a proper python
> script that contains the actual unit tests of fedabipkgdiff, rather than
> having the unit tests be in the fedabipkgdiff file itself.
>
> That file would then load (using the imp.load_source() function from the
> python standard library) by specifying the path to the the fedabipkgdiff
> from the source, so that "make check" uses the fedabipkgdiff from the
> sources and not the one that might already be installed on the
> system. There is a potentially a little bit of autoconf magic to do
> here, but I can help you here for the details.
I finally did it. I am attaching the patch at the end of this message.
The patch is also available in my "dodji/fedabipkgdiff" branch at https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commitdiff;h=246f4f3280d8b96911d4ef2676f0d46ff66b0681
What do you think?
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: the patch --]
[-- Type: text/x-patch, Size: 17035 bytes --]
From 246f4f3280d8b96911d4ef2676f0d46ff66b0681 Mon Sep 17 00:00:00 2001
From: Dodji Seketeli <dodji@redhat.com>
Date: Sat, 20 Feb 2016 12:42:04 +0100
Subject: [PATCH 3/3] Move unit tests from fedabipkgdiff to
runtestfedabipkgdiff.py
The fedabipkgdiff program used to contain unit tests for many of its
functions. This patch moves these unit tests out of the
tools/fedabipkgdiff program file, to the IMHO more appropriate
location tests/runtestfedabipkgdiff.py. The later file starts by
loading the tools/fedabipkgdiff program. The unit test code it
contains then tests the functions of the program that has just been
loaded. I believe this separation decreases the clutter of
fedabipkgdiff and thus increases its maintainability.
Note that tests/runtestfedabipkgdiff.py is now generated by autoconf
inside the build directory. To do that, autoconf takes the pattern
file tests/runtestfedabipkgdiff.py.in from the source directory and
replaces variables like @top_srcdir@ by their actual value. The
result is a tests/runtestfedabipkgdiff.py file which is a proper
executable python program that is executed when the user invokes the
standard "make check" command from the build directory. Of course,
parallel checking still works too, by doing "make -j8 check" if your
system has eight cores, for instance.
The patch passes "make distcheck" too.
* configure.ac: Add check for prerequisite python modules
itertools, shutil, unittest and mock. These are necessary for the
unit test of fedabipkgdiff. Generate
tests/runtestfedabipkgdiff.py into the build directory, from the
tests/runtestfedabipkgdiff.py.in input file.
* tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
autoconf template file in here.
* tests/runtestfedabipkgdiff.py.in: New unit test file.
* tests/runtestfedabipkgdiff.sh: Remove.
Signed-off-by: Dodji Seketeli <dodji@redhat.com>
---
configure.ac | 9 +-
tests/Makefile.am | 8 +-
tests/runtestfedabipkgdiff.py.in | 335 +++++++++++++++++++++++++++++++++++++++
tests/runtestfedabipkgdiff.sh | 5 -
4 files changed, 347 insertions(+), 10 deletions(-)
create mode 100755 tests/runtestfedabipkgdiff.py.in
delete mode 100755 tests/runtestfedabipkgdiff.sh
diff --git a/configure.ac b/configure.ac
index 687a2ca..17bfe30 100644
--- a/configure.ac
+++ b/configure.ac
@@ -285,7 +285,11 @@ if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
AX_PYTHON_MODULE(sys, $FATAL, python2)
AX_PYTHON_MODULE(itertools, $FATAL, python2)
AX_PYTHON_MODULE(urlparse, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(shutil, $FATAL, python2)
+ AX_PYTHON_MODULE(unittest, $FATAL, python2)
AX_PYTHON_MODULE(koji, $FATAL, python2)
+ AX_PYTHON_MODULE(mock, $FATAL, python2)
ENABLE_FEDABIPKGDIFF=yes
if test x$ENABLE_FEDABIPKGDIFF != xyes; then
@@ -437,7 +441,10 @@ libabigail.pc
bash-completion/Makefile])
dnl Some test scripts are generated by autofoo.
-AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh], [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh],
+ [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py],
+ [chmod +x tests/runtestfedabipkgdiff.py])
AC_OUTPUT
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 958995c..953dfef 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,10 +31,10 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
-runtestfedabipkgdiff.sh \
+runtestfedabipkgdiff.py \
$(CXX11_TESTS)
-EXTRA_DIST = runtestcanonicalizetypes.sh.in
+EXTRA_DIST = runtestcanonicalizetypes.sh.in runtestfedabipkgdiff.py.in
CLEANFILES = \
runtestcanonicalizetypes.output.txt \
runtestcanonicalizetypes.output.final.txt
@@ -115,8 +115,8 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
-runtestfedabipkgdiff_sh_SOURCES =
-runtestfedabipkgdiff.sh$(EXEEXT):
+runtestfedabipkgdiff_py_SOURCES =
+runtestfedabipkgdiff.py$(EXEEXT):
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
new file mode 100755
index 0000000..7842cc9
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -0,0 +1,335 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library. This program is free
+# software; you can redistribute it and/or modify it under the terms
+# of the GNU General Public License as published by the Free Software
+# Foundation; either version 3, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this program; see the file COPYING-LGPLV3. If
+# not, see <http://www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import os
+import itertools
+import shutil
+import unittest
+
+try:
+ from mock import patch
+except ImportError:
+ print >>sys.stderr, \
+ 'mock is not installed. Please install it before running tests.'
+ sys.exit(1)
+
+import imp
+# Import the fedabipkgdiff program file from the source directory.
+fedabidiff_mod = imp.load_source('fedabidiff',
+ '@top_srcdir@/tools/fedabipkgdiff')
+
+counter = itertools.count(0)
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_fedora_distro(self):
+ distro = 'fc5'
+ self.assertTrue(fedabidiff_mod.is_fedora_distro(distro))
+
+ distro = 'f5'
+ self.assertFalse(fedabidiff_mod.is_fedora_distro(distro))
+
+ distro = 'fc23'
+ self.assertTrue(fedabidiff_mod.is_fedora_distro(distro))
+
+ distro = 'fc'
+ self.assertFalse(fedabidiff_mod.is_fedora_distro(distro))
+
+ distro = 'fc234'
+ self.assertFalse(fedabidiff_mod.is_fedora_distro(distro))
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff"""
+
+ def setUp(self):
+ self.pkg1_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_single_info = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg1_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ self.pkg2_infos = {
+ 'i686': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'x86_64': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ 'armv7hl': [{'downloaded_rpm_file': 'dummy file path'},
+ {'downloaded_rpm_file': 'dummy file path'}],
+ }
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 0
+
+ result = fedabidiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = fedabidiff_mod.run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ mock_abipkgdiff.return_value = 4
+
+ result = fedabidiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = fedabidiff_mod.run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('fedabidiff.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ result = fedabidiff_mod.run_abipkgdiff(self.pkg1_infos, self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+fake_rpm_file = 'foo-0.1-1.fc24.x86_64.rpm'
+fake_debuginfo_rpm_file = 'foo-debuginfo-0.1-1.fc24.x86_64.rpm'
+
+
+class MockGlobalConfig(object):
+ koji_server = fedabidiff_mod.DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ return MockKojiClientSession(baseurl=fedabidiff_mod.DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock"""
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class SelectRpmsFromABuildTest(unittest.TestCase):
+ """Test case for select_rpms_from_a_build"""
+
+ def assert_rpms(self, rpms):
+ for item in rpms:
+ self.assertTrue(item['arch'] in ['i686', 'x86_64'])
+ self.assertTrue(item['name'] in ('httpd', 'httpd-debuginfo'))
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_all_arches(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabidiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+ @patch('fedabidiff.Brew.listRPMs')
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ def test_select_rpms_from_one_arch(self, mock_listRPMs):
+ mock_listRPMs.return_value = [
+ {'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ ]
+
+ session = fedabidiff_mod.get_session()
+ rpms = session.select_rpms_from_a_build(1, 'httpd')
+ self.assert_rpms(rpms)
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ session = fedabidiff_mod.get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_fail_to_find_latest_build(self):
+ session = fedabidiff_mod.get_session()
+ latest_build = session.get_package_latest_build('httpd', 'xxxx')
+ self.assertEquals(None, latest_build)
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('fedabidiff.koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ session = fedabidiff_mod.get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+class FindLocalDebuginfoRPMTest(unittest.TestCase):
+ """Test case for find_local_debuginfo_rpm"""
+
+ def setUp(self):
+ # FIXME: is it possible to patch glob or the underlying methods glob
+ # depends on
+ self.test_dir = './test_dir'
+ os.makedirs(self.test_dir)
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir, fake_debuginfo_rpm_file)))
+ os.system('touch {0}'.format(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file.replace('foo', 'another'))))
+
+ def tearDown(self):
+ shutil.rmtree(self.test_dir)
+
+ def test_find_debuginfo_rpm(self):
+ debuginfo_rpm = fedabidiff_mod.find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, fake_rpm_file))
+
+ expected_debuginfo_rpm_file = os.path.abspath(
+ os.path.join(self.test_dir,
+ fake_debuginfo_rpm_file))
+ self.assertEquals(expected_debuginfo_rpm_file, debuginfo_rpm)
+
+ def test_no_suitable_debuginfo_rpm(self):
+ debuginfo_rpm = fedabidiff_mod.find_local_debuginfo_rpm(
+ os.path.join(self.test_dir, 'abc-0.1-1.i686.rpm'))
+ self.assertEquals(None, debuginfo_rpm)
+
+
+invoked_from_cmd = __name__ == '__main__'
+
+if invoked_from_cmd:
+ unittest.main()
diff --git a/tests/runtestfedabipkgdiff.sh b/tests/runtestfedabipkgdiff.sh
deleted file mode 100755
index 4144419..0000000
--- a/tests/runtestfedabipkgdiff.sh
+++ /dev/null
@@ -1,5 +0,0 @@
-#!/usr/bin/bash
-
-export FEDABIPKGDIFF_TESTS=1
-
-../../tools/fedabipkgdiff
\ No newline at end of file
--
1.8.3.1
[-- Attachment #3: Type: text/plain, Size: 13 bytes --]
--
Dodji
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Chenxiong Qi
@ 2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Chenxiong Qi
0 siblings, 1 reply; 10+ messages in thread
From: Dodji Seketeli @ 2016-01-01 0:00 UTC (permalink / raw)
To: Chenxiong Qi; +Cc: libabigail
Hello Chenxiong,
Sorry for my being late to review this. It just felt through the cracks
while I was looking as something else. Oh well.
So I reviewed it and I must say this is good stuff :-)
I looked at it in details and the comments I have now are mostly about
documentations and the like.
Please find my comments below.
> * autoconf-archive/ax_compare_version.m4: New file copied from the
Each of these lines should start with a 'tab' character (you know, the
\t character), as specified in the file COMMIT-LOG-GUIDELINES:
The subsequent lines should have the form of the Body of a GNU ChangeLog
entry, i.e:
* file1.c (func1): Changed foo in this function.
(func2): Changed blah in that function
* file2.c (func_foo): Changed something here.
Note that before the '*', there is a tab that is 8 spaces long. Also
note that right after the '*', there is a space.
To ease the task, I have edited your ChangeLog. Here is the result,
that you can add to your patch:
* autoconf-archive/ax_compare_version.m4: New file copied from the
autoconf-archive project.
* autoconf-archive/ax_prog_python_version.m4: Likewise.
* autoconf-archive/ax_python_module.m4: Likewise.
* Makefile.am: Add the new files above to the source distribution.
* configure.ac: Include the new m4 macros from the autoconf
archive. Add a new --enable-fedabipkgdiff option. Update the
report at the end of the configure process to show the status of
the fedabipkgdiff feature. Add check for prerequisite python
modules itertools, shutil, unittest and mock. These are necessary
for the unit test of fedabipkgdiff. Generate
tests/runtestfedabipkgdiff.py into the build directory, from the
tests/runtestfedabipkgdiff.py.in input file.
* tools/Makefile.am: Include the fedabipkgdiff to the source
distribution and install it if the "fedabipkgdiff" feature is
enabled.
* tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
autoconf template file in here.
* tests/runtestfedabipkgdiff.py.in: New unit test file.
* tools/fedabipkgdiff: New tool fedabipkgdiff.
[...]
> diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
> new file mode 100755
> index 0000000..3087212
> --- /dev/null
> +++ b/tests/runtestfedabipkgdiff.py.in
> @@ -0,0 +1,450 @@
> +#!/usr/bin/python
> +# -*- coding: utf-8 -*-
> +# -*- Mode: Python
> +#
> +# This file is part of the GNU Application Binary Interface Generic
> +# Analysis and Instrumentation Library. This program is free
> +# software; you can redistribute it and/or modify it under the terms
> +# of the GNU General Public License as published by the Free Software
> +# Foundation; either version 3, or (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful, but
> +# WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# General Lesser Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this program; see the file COPYING-LGPLV3. If
> +# not, see <http://www.gnu.org/licenses/>.
This is a program, not a library. In the project, by default,
programs are GPL. It's only the library that is LGPL. So if this is
an oversight, I guess we should fix it and change this license to GPL.
> +#
> +# Author: Chenxiong Qi
> +
> +import os
> +import itertools
> +import shutil
> +import unittest
> +
> +import koji
In general, I find that this file lacks comments. There should be a
general comment at the beginning of the file that introduces the kind
of tests that are performed here. For instance,
tests/test-diff-pkg.cc has:
/// @file
///
/// This test harness program computes the ABI changes between ELF
/// binaries present inside input packages. Some of the input
/// packages have debuginfo, some don't. The resulting ABI change
/// report is then compared with a reference one.
///
/// The set of input files and reference reports to consider should be
/// present in the source distribution, which means they must be
/// referenced in tests/data/Makefile.am by the EXTRA_DIST variable.
And if you look into that file, there are various other comments that
describe how the test is organized etc. I think we should try and do
the same for all the tests that we add.
For instance:
[...]
> +counter = itertools.count(0)
What is this global variable for? We should have a comment for this.
> +class UtilsTest(unittest.TestCase):
We should have an introductory comment for this, I believe.
> +
> + def test_is_fedora_distro(self):
Same for this test. What is the philosophy behind this? What is the
kind of "distro strings" we want to accept? etc.
> + distro = 'fc5'
> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'f5'
> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'fc23'
> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'fc'
> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'fc234'
> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'el7'
> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> + distro = 'el7_2'
> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
> +
> +
> +class RPMTest(unittest.TestCase):
> + """Test case for RPM class"""
For instance, I would have expected this comment to say something
introductory about the tests performed on the fedabipkgdiff.RPM
class.
> +
> + def setUp(self):
Here, I would have expected a comment about what this
debuginfo_rpm_info data member is, saying that it must be the same
type as the parameter type expected by the constructor of the
fedabipkgdiff.RPM class. So if that constructor's parameter type
changes, this test must be updated. Something like that.
> + self.debuginfo_rpm_info = {
> + 'arch': 'i686',
> + 'name': 'httpd-debuginfo',
> + 'release': '1.fc22',
> + 'version': '2.4.18'
> + }
Likewise.
> + self.rpm_info = {
> + 'arch': 'x86_64',
> + 'name': 'httpd',
> + 'release': '1.fc22',
> + 'version': '2.4.18'
> + }
> +
> + def test_attribute_access(self):
Please add a comment saying what this test does. At least its
intent. For instance:
This test enforces the exported interface of the fedabipkgdiff.RPM
class. If a new data member is added to that type, removed or
changed in that class, please update this test accordingly.
Maybe repeat here that the data members of the RPM class are the same
as the data members of the poorly documented return type of the koji
getRPM request, and so whenever that return type changes, we need to
update this test. You know, something like that. Otherwise, there is
too much implicit knowledge here, and this makes maintenance painful
for anyone would would have to take over tomorrow. And this is
especially true for a program in python where types are so loosely
defined and thus where type mis-matches won't be caught until the
program's runtime.
etc.
> + rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
> + self.assertEquals(self.debuginfo_rpm_info['arch'], rpm.arch)
> + self.assertEquals(self.debuginfo_rpm_info['name'], rpm.name)
> + self.assertEquals(self.debuginfo_rpm_info['release'], rpm.release)
> + self.assertEquals(self.debuginfo_rpm_info['version'], rpm.version)
[...]
I think all test cases should be documented similarly. And also, all
the classes that are helper classes for the tests should be
documented.
> --- /dev/null
> +++ b/tools/fedabipkgdiff
> @@ -0,0 +1,805 @@
> +#!/usr/bin/env python
> +# -*- coding: utf-8 -*-
> +# -*- Mode: Python
> +#
> +# This file is part of the GNU Application Binary Interface Generic
> +# Analysis and Instrumentation Library. This program is free
> +# software; you can redistribute it and/or modify it under the terms
> +# of the GNU General Public License as published by the Free Software
> +# Foundation; either version 3, or (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful, but
> +# WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# General Lesser Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this program; see the file COPYING-LGPLV3. If
> +# not, see <http://www.gnu.org/licenses/>.
Likewise, I believe this should be GPL.
> +#
> +# Author: Chenxiong Qi
> +
Please add an introductory comment here, explaining the intend of the
tool. Just like what is done for in tools/abidw.cc:
/// @file
///
/// This program reads an elf file, try to load its debug info (in
/// DWARF format) and emit it back in a set of "text sections" in native
/// libabigail XML format.
[...]
Generally speaking, I would like to see all the function parameter
*types* and return values to be documented.
[...]
> +def log_call(func):
> + def proxy(*args, **kwargs):
Please add a comment that says that this is a decorator. And please
tell what we expect *args and **kwargs to be. (the arguments of func,
as well as the key word arguments, if any).
[...]
> +class RPM(object):
> + """Represeting a RPM"""
> +
> + def __init__(self, data):
> + """Initialize a RPM object
> +
> + :param dict data: a dict representing a RPM information got from koji
> + API, either listRPMs or getRPM
What are the keys and values expected in that dict? Granted the koji
API is so "lightly" documented, it doesn't really help. But I think
we could write an example of the dict keys/values that *we* expect
here, to ease maintenance.
Also, you could say something about the fact that the set of data
members expected by this class is enforced in the unit test case
[...]
> + @property
> + def nvra(self):
> + return '%(name)s-%(version)s-%(release)s.%(arch)s' % self.data
> +
> + @property
> + def filename(self):
> + return '{0}.rpm'.format(self.nvra)
> +
> + @property
> + def is_debuginfo(self):
> + """Check if a RPM is a debuginfo"""
I would have said:
"Check if the name of the current RPM denotes a debug info
package"
> + return koji.is_debuginfo(self.data['name'])
> +
> + @property
> + def download_url(self):
> + """Get the URL from where to download from koji"""
> + build = session.getBuild(self.build_id)
> + return os.path.join(pathinfo.build(build), pathinfo.rpm(self.data))
> +
> + @property
> + def downloaded_file(self):
> + """Get a pridictable downloaded file name with absolute path"""
> + # arch should be removed from the result returned from PathInfo.rpm
> + filename = os.path.basename(pathinfo.rpm(self.data))
> + return os.path.join(get_download_dir(), filename)
> +
> + @property
> + def is_downloaded(self):
Please add a comment here.
> + return os.path.exists(self.downloaded_file)
> +
> +
> +class LocalRPM(RPM):
> + """Representing a local RPM
> +
> + Local RPM means the one that could be already downloaded or built from
> + where I can find it
> + """
> +
> + def __init__(self, filename):
Please add a comment here. What is the type of the "filename"
parameter etc...
> + self.local_filename = filename
> + self.data = koji.parse_NVRA(os.path.basename(filename))
> +
> + @property
> + def downloaded_file(self):
Comments.
> +
> +class Brew(object):
> + """Proxy to kojihub XMLRPC with additional extensions to fedabipkgdiff
> +
> + kojihub XMLRPC APIs are well-documented in koji's source code. For more
> + details information, please refer to class RootExports within kojihub.py.
> + """
Please add an html link to the koji source code where the XMLRPC APIs
are documented.
> +
> + def __init__(self, baseurl):
> + """Initialize Brew that is a proxy to koji.ClientSession"""
> + self.session = koji.ClientSession(baseurl)
> +
> + @log_call
> + def listRPMs(self, selector=None, **kwargs):
> + """Proxy to kojihub.listRPMs
> +
> + :param selector: to adjust if a RPM should be selected
> + :type selector: a callable object
> + :param kwargs: keyword parameters accepted by kojihub.listRPMs
Here if you could give examples of the keywords (and their arguments)
that we'd pass, it would be good. I know the reader can always go dig
into kojihub.listRPMs, but examples would help. I think redundancy is
good here.
> + :type kwargs: dict
> + :return: a list of RPMs, each of them is a dict object
Again, an example of the dict object that we expect would be good.
> + :rtype: list
> + """
> + if selector:
> + assert hasattr(selector, '__call__'), 'selector should be callable'
> + rpms = self.session.listRPMs(**kwargs)
> + if selector:
> + rpms = [rpm for rpm in rpms if selector(rpm)]
> + return rpms
> +
> + @log_call
> + def getRPM(self, *args, **kwargs):
Comments.
> + rpm = self.session.getRPM(*args, **kwargs)
> + if rpm is None:
> + raise RpmNotFound('Cannot find RPM {0}'.format(args[0]))
> + return rpm
> +
> + @log_call
> + def listBuilds(self, topone=None, selector=None, order_by=None,
> + reverse=None, **kwargs):
> + """Proxy to kojihub.listBuilds to list completed builds
Please document here (yeah I know it's redundant) the parameters that
kojihub.listBuilds expects.
> +
> + Suport additional two keyword parameters:
> +
> + :param bool topone: whether to return the top first one
> + :param selector: a callable object used to select specific subset of
> + builds
> + :type selector: callable object
> + :param str order_by: the attribute name by which to order the builds,
> + for example, name, version, or nvr.
> + :param bool reverse: whether to order builds reversely
> + :param dict kwargs: keyword parameters accepted by kojihub.listBuilds
> + :return: a list of builds, even if just return only one build
> + :rtype: list
> + """
> + if 'state' not in kwargs:
> + kwargs['state'] = koji.BUILD_STATES['COMPLETE']
> +
> + if selector is not None and not hasattr(selector, '__call__'):
> + raise TypeError(
> + '{0} is not a callable object.'.format(str(selector)))
> +
> + if order_by is not None and not isinstance(order_by, basestring):
> + raise TypeError('order_by {0} is invalid.'.format(order_by))
> +
> + builds = self.session.listBuilds(**kwargs)
> + if selector is not None:
> + builds = [build for build in builds if selector(build)]
> + if order_by is not None:
> + # FIXME: is it possible to sort builds by using opts parameter of
> + # listBuilds
> + builds = sorted(builds,
> + key=lambda item: item[order_by],
> + reverse=reverse)
> + if topone:
> + builds = builds[0:1]
> +
> + return builds
> +
> + @log_call
> + def getPackage(self, name):
> + """Proxy to kojihub.getPackage
> +
> + :param str name: package name
Please give example of the package name that we expect, so that a user
of this function has an idea of the correct form expected.
> + :return: a dict object representing a package
> + :rtype: dict
> + """
> + package = self.session.getPackage(name)
> + if package is None:
> + package = self.session.getPackage(name.rsplit('-', 1)[0])
> + if package is None:
> + raise KojiPackageNotFound(
> + 'Cannot find package {0}.'.format(name))
> + return package
> +
> + @log_call
> + def getBuild(self, *args, **kwargs):
> + """Proxy to kojihub.getBuild"""
Please describe the (keyword) arguments expected.
> + return self.session.getBuild(*args, **kwargs)
> +
> + @log_call
> + def get_rpm_build_id(self, name, version, release, arch=None):
> + """Get build ID that contains a rpm with specific nvra
> +
> + If arch is omitted, a rpm can be identified easily by its N-V-R-A.
> +
> + If arch is omitted, name is used to get associated package, and then
> + to get the build.
> +
> + :param str name: name of a rpm
> + :param str version: version of a rpm
> + :param str release: release of a rpm
I think that here, example of what you expect as release (at least)
would be interesting.
[...]
> +
> + @log_call
> + def get_package_latest_build(self, package_name, distro):
> + """Get latest build from a package
> +
> + :param str package_name: from which package to get the latest build
> + :param str distro: which distro the latest build belongs to
The form of what is expected as distro would be important too. That
would help to maintain the test suite.
[...]
+@log_call
+def get_session():
Comments.
+ return Brew(global_config.koji_server)
[...]
+def get_download_dir():
+ """Return the directory holding all downloaded rpms"""
+ download_dir = os.path.join(HOME_DIR, 'downloads')
I would use the standard $XDG_CACHE_HOME place, to store the
downloaded packages:
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html.
That is, ~/.local/fedabipkgdiff.
I could also use a fedabipkgdiff --clean-cache option to clean the
cache cache, but we can add that this later, after the patch is in.
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
> +@log_call
> +def abipkgdiff(pkg_info1, pkg_info2):
> + """Run abipkgdiff against found two RPM packages"""
Please document the expected types of the parameters here.
+ print 'ABI check on {0} and {1}'.format(pkg_info1.package.filename,
+ pkg_info2.package.filename)
Please, rather print:
'Comparing the ABI of binaries between {0} and {1}:'
[...]
> +def magic_construct(rpms):
> + """Construct RPMs into a magic structure
> +
> + Convert list of
> +
> + foo-1.0-1.fc22.i686
> + foo-debuginfo-1.0-1.fc22.i686
> + foo-devel-1.0-1.fc22.i686
> +
> + to list of
> +
> + (foo-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
> + (foo-devel-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
> + """
> + debuginfo = None
> + packages = []
> + for rpm in rpms:
> + if rpm.is_debuginfo:
> + debuginfo = rpm
> + else:
> + packages.append(rpm)
> + return [PkgInfo(package, debuginfo) for package in packages]
> +
> +
> +@log_call
> +def run_abipkgdiff(pkg1_infos, pkg2_infos):
> + """Run abipkgdiff
> +
> + If one of the executions finds ABI differences, the return code is the
> + return code from abipkgdiff.
> +
> + :param dict pkg1_infos: a dict mapping from arch to list of rpms, that is
> + returned from method make_rpms_usable_for_abipkgdiff
Rather than refer to what make_rpms_usable_for_abipkgdiff returns,
please redundantly describe the dict here, giving a real example, so
that a user willing to use this function can see what she should feed
it with.
[...]
> +@log_call
> +def diff_local_rpm_with_latest_rpm_from_koji():
> + """Diff against local rpm and remove latest rpm
> +
> + This operation handles a local rpm and debuginfo rpm and remote ones
> + located in remote Koji server, that has specific distro specificed by
> + argument --from.
> +
> + 1/ Suppose the packager has just locally built a package named
> + foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
> + latest stable package from Fedora 23, one would do:
> +
> + fedabipkgdiff --from f23 ./foo-3.0.fc24.rpm
> + """
Great comment, thanks.
[...]
> +@log_call
> +def make_rpms_usable_for_abipkgdiff(rpms):
> + """
> + Construct result that contains mappings from arch to download url and
> + downloaded rpm filename of rpm and debuginfo rpm
> +
> + :return: a mapping from an arch to a list of rpms
Please give an example of this dict.
[...]
> +@log_call
> +def diff_rpms_with_nvra(name, version, release, arch=None,
> + all_subpackages=None):
Comments.
> + build_id = session.get_rpm_build_id(name, version, release, arch)
> + rpms = session.select_rpms_from_a_build(build_id, name, arches=arch,
> + select_subpackages=all_subpackages)
> + return make_rpms_usable_for_abipkgdiff(rpms)
> +
Also, I wouldn't call this function "diff_rpms_with_nvra", because the
function doesn't perform any diffing, does it? Rather, my
understanding is that it constructs the two sets of package
descriptors that would be passed to abipkgdiff.
> +@log_call
> +def diff_two_nvras_from_koji():
> + """Diff two nvras from koji
> +
> + The arch probably omits, that means febabipkgdiff will diff all arches. If
I would say "the arch is probably omitted ..."
[...]
+def main():
[...]
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to -h.'
+ returncode = 1
Please, refer to --help, rather -h just like what is done in the other
libabigail tools.
+
+ return returncode
All in all, I am wondering if we shouldn't add some progress
indication to the user, just like what "dnf" does. You know, saying
things like "we are downloading this package etc, and now we are
comparing these packages, etc." Otherwise, the time taken can seem
quite long. But we can add this later, after the patch has been
committed, I guess.
Also, we should arrange for the downloads to happen in parallel,
somehow. Again, this can happen after the patch is in.
Now, the other big thing missing from this patch is a manual.
That is really important. Every single libabigail tool has a
documentation in docs/manuals. Documentation is in the restructured
text format, and we use python-sphinx to generate html, man and info
variants for it. The manual should also say where the packages are
downloaded.
If you don't feel like adding a manual for this tool, please tell me,
I'll help.
I love this patch very much! Thank you for your hard work on this.
This is simply awesome. I cannot wait for it to get in :-)
Cheers,
--
Dodji
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Chenxiong Qi
@ 2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Chenxiong Qi
0 siblings, 1 reply; 10+ messages in thread
From: Dodji Seketeli @ 2016-01-01 0:00 UTC (permalink / raw)
To: Chenxiong Qi; +Cc: Chenxiong Qi, libabigail
Hello Chenxiong,
[...]
> This time, the patch is updated
>
> - more documentation inside fedabipkgdiff and runtestfedabipkgdiff.py.in
> - using curl instead of wget to download packages from Koji
> - a new manual (with Dodji's help, man page can be generated from this manual by sphnix)
> - rebased, on latest commit fa5a5acbbcb1e
Thank you for all these efforts!
So I have reviewed this updated patch, made some changes and committed
the result to the master branch of the git repository. Woohoo :-)
Below are some comments about the parts of your patch where I made
some changes. If you think we should amend them, please do not
hesitate to say it.
[...]
> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
[...]
> +import xdg.BaseDirectory
You are importing a new python module (xdg) that needs to be installed
otherwise the execution of the fedabipkgdiff program just halts here.
We thus need to ensure that the configure scripts detects when the
python module is not installed, and gives a hint to the user saying
that she needs to install it. So I added this hunk:
--- a/configure.ac
+++ b/configure.ac
@@ -288,6 +288,7 @@ if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
AX_PYTHON_MODULE(itertools, $FATAL, python2)
AX_PYTHON_MODULE(shutil, $FATAL, python2)
AX_PYTHON_MODULE(unittest, $FATAL, python2)
+ AX_PYTHON_MODULE(xdg, $FATAL, python2)
AX_PYTHON_MODULE(koji, $FATAL, python2)
AX_PYTHON_MODULE(mock, $FATAL, python2)
ENABLE_FEDABIPKGDIFF=yes
[...]
> diff --git a/doc/manuals/Makefile.am b/doc/manuals/Makefile.am
> index f23d1ab..5128bc0 100644
> --- a/doc/manuals/Makefile.am
> +++ b/doc/manuals/Makefile.am
> @@ -10,7 +10,8 @@ conf.py \
> index.rst \
> libabigail-concepts.rst \
> libabigail-overview.rst \
> -libabigail-tools.rst
> +libabigail-tools.rst \
> +fedabipkgdiff
Here there is no file named 'fedabipkgdiff' to be added to the set of
files that are doing to be distributed in the tarball that is
constructed by the 'make dist' command.
What you want is to add the file fedabipkgdiff.rst. So I have added
this the hunk below that fixes that.
diff --git a/doc/manuals/Makefile.am b/doc/manuals/Makefile.am
index 5128bc0..573fb3b 100644
--- a/doc/manuals/Makefile.am
+++ b/doc/manuals/Makefile.am
@@ -11,7 +11,7 @@ index.rst \
libabigail-concepts.rst \
libabigail-overview.rst \
libabigail-tools.rst \
-fedabipkgdiff
+fedabipkgdiff.rst
Note that you could have detected the issue by typing the command:
make distcheck
which checks that constructing the tarball works correctly. That
'distcheck' target goes even further and tries to *compile* the
tarball that was successfully built and runs all the tests. When
"make distcheck" works, it means the software is ready to be released
:-) We should always run make distcheck before submitting a patch, I
believe.
When looking at the tools/Makefile.am file I noticed that I forgot to
add the fedabipkgdiff to the set of files that needs to be
*distributed*. It was just meant to be installed in the /usr/bin/
directory. I have thus added the hunk below to fix that:
diff --git a/tools/Makefile.am b/tools/Makefile.am
index 0d96215..3e53eb1 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -7,7 +7,7 @@ else
endif
if ENABLE_FEDABIPKGDIFF
- bin_SCRIPTS = fedabipkgdiff
+ dist_bin_SCRIPTS = fedabipkgdiff
else
noinst_SCRIPTS = fedabipkgdiff
endif
[...]
> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
> +HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home,
> + os.path.splitext(os.path.basename(__file__))[0])
As I said in my previous review, I prefer this to be XDG_CACHE_HOME,
rather than XDG_DATA_HOME. The reason being that I consider the
downloaded packages as being cached data, rather than application
data. So I changed this to XDG_CACHE_HOME.
The resulting hunk is:
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
index e7c4785..f2b20be 100755
--- a/tools/fedabipkgdiff
+++ b/tools/fedabipkgdiff
@@ -60,7 +60,7 @@ DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
# The working directory where to hold all data including downloaded RPM
# packages Currently, it's not configurable and hardcode here. In the future
# version of fedabipkgdiff, I'll make it configurable by users.
-HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home,
+HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_cache_home,
os.path.splitext(os.path.basename(__file__))[0])
# Used to construct abipkgdiff command line argument, package and
associated
[...]
> + def listBuilds(self, packageID, state=None, topone=None,
> + selector=None, order_by=None, reverse=None):
[...]
> + builds = self.session.listBuilds(packageId=packageID, state=state)
Here, packageId should be packageID instead. And this is causing a
runtime error. So I fixed it. The resulting hunk is:
@@ -417,7 +417,7 @@ class Brew(object):
if order_by is not None and not isinstance(order_by, basestring):
raise TypeError('order_by {0} is invalid.'.format(order_by))
- builds = self.session.listBuilds(packageId=packageID, state=state)
+ builds = self.session.listBuilds(packageID=packageID, state=state)
if selector is not None:
builds = [build for build in builds if selector(build)]
if order_by is not None:
[...]
> +def download_rpm(url):
[...]
> + cmd = 'curl --silent -C - -O {} > {}'.format(
> + url, os.path.join(get_download_dir(),
> + os.path.basename(url)))
The resulting rpm file named
os.path.join(get_download_dir(), os.path.basename(url))
are empty here. So the comparison performed by abipkgdiff is failing
on my system. This is because when you use the -O option, curl
doesn't emit anything on standard output, at least on the curl 7.29.0
version that I am using on my system.
Also, I think the -C option is not critically useful here.
Furthermore, I noticed that this program tries to download packages
even when the --dry-run option is used. I think it's useful to avoid
downloading packages when --dry-run is turned on.
So I changed this using this hunk:
@@ -621,9 +621,13 @@ def download_rpm(url):
:return: True if a RPM is downloaded successfully, False otherwise.
:rtype: bool
"""
- cmd = 'curl --silent -C - -O {} > {}'.format(
+ cmd = 'curl --silent {} -o {}'.format(
url, os.path.join(get_download_dir(),
os.path.basename(url)))
+ if global_config.dry_run:
+ print 'DRY-RUN:', cmd
+ return
+
return_code = subprocess.call(cmd, shell=True)
if return_code > 0:
logger.error('curl fails with returned code: %d.', return_code)
[...]
> +def abipkgdiff(pkg_info1, pkg_info2):
[...]
> + cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
> + pkg_info1.debuginfo_package.downloaded_file,
> + pkg_info2.debuginfo_package.downloaded_file,
> + pkg_info1.package.downloaded_file,
> + pkg_info2.package.downloaded_file)
So here fedabipkgdiff is comparing the ABI of *all* binaries in the
packages, including executable binaries. By default I think it makes
more sense to only compare the ABI of shared libraries, not executable
binaries. And it's faster too. So I changed this by doing:
@@ -672,7 +672,7 @@ def abipkgdiff(pkg_info1, pkg_info2):
:return: return code of underlying abipkgdiff execution.
:rtype: int
"""
- cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ cmd = 'abipkgdiff --dso-only --d1 {0} --d2 {1} {2} {3}'.format(
pkg_info1.debuginfo_package.downloaded_file,
pkg_info2.debuginfo_package.downloaded_file,
pkg_info1.package.downloaded_file,
I made wording changes in the build_commandline_args_parser function.
The resulting hunk is:
@@ -924,7 +928,7 @@ def diff_two_nvras_from_koji():
def build_commandline_args_parser():
parser = argparse.ArgumentParser(
- description='Run abipkgdiff against RPM packages from koji')
+ description='Compare ABI of shared libraries in RPM packages from the Koji build system')
parser.add_argument(
'NVR',
@@ -936,20 +940,20 @@ def build_commandline_args_parser():
required=False,
dest='dry_run',
action='store_true',
- help='Don\'t actually run abipkgdiff. The commands that should be '
+ help='Don\'t actually do the work. The commands that should be '
'run will be sent to stdout.')
parser.add_argument(
'--from',
required=False,
metavar='DISTRO',
dest='from_distro',
- help='baseline Fedora distro, for example, fc23')
+ help='baseline Fedora distribution name, for example, fc23')
parser.add_argument(
'--to',
required=False,
metavar='DISTRO',
dest='to_distro',
- help='which Fedora distro to compare, for example, fc24')
+ help='Fedora distribution name to compare against the baseline, for example, fc24')
parser.add_argument(
'-a',
'--all-subpackages',
After I started to use the global_config.dry_run member variable in
download_rpm, I had to update the unit tests that exercise that member
function in tests/runtestfedabipkgdiff.py.in.
To do so, I had to move the test class DownloadRPMTest to *after* the
class MockGlobalConfig is defined, so that member functions of
DownloadRPMTest can use the MockGlobalConfig, which is now needed by
the member functions of DownloadRPMTest that exercise the
download_rpm function. Of course, I had to add a dry_run data member
to MockGlobalConfig and update the comment in there accordingly.
The resulting hunk is:
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
index 84a4320..483cfdb 100755
--- a/tests/runtestfedabipkgdiff.py.in
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -410,64 +410,6 @@ class RunAbipkgdiffTest(unittest.TestCase):
self.pkg2_infos)
self.assertTrue(result > 0)
-
-class DownloadRPMTest(unittest.TestCase):
- """Test case for download_rpm
-
- Download a remote file, which is a local file simulating a remote file with
- scheme file://, for example file:///tmp/a.txt, to download directory.
- """
-
- def setUp(self):
- # Create a remote file for testing download of this file
- self.fd, self.remote_filename = tempfile.mkstemp(
- prefix=temp_file_or_dir_prefix)
- # Whatever the content is, this case does not care about. Close it
- # immediately.
- os.close(self.fd)
-
- # Used as a fake download directory to mock get_download_dir method
- self.download_dir = tempfile.mkdtemp(prefix=temp_file_or_dir_prefix)
-
- def tearDown(self):
- os.remove(self.remote_filename)
- shutil.rmtree(self.download_dir)
-
- def make_remote_file_url(self):
- """Make URL of remote file that is used for downloading this file"""
- return 'file://{}'.format(self.remote_filename)
-
- def make_nonexistent_remote_file_url(self):
- """Return URL to a non-existent remote file"""
- return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
-
- @patch('__main__.fedabipkgdiff_mod.get_download_dir')
- def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
- """Enusre True is returned if curl succeeds to download remote file
-
- Download remote file to a fake download directory. Ensure everything is
- okay, and return value from download_rpm should be truth.
- """
- mock_get_download_dir.return_value = self.download_dir
-
- url = self.make_remote_file_url()
- ret = fedabipkgdiff_mod.download_rpm(url)
- self.assertTrue(ret)
-
- @patch('__main__.fedabipkgdiff_mod.get_download_dir')
- def test_failed_to_download_a_rpm(self, mock_get_download_dir):
- """Ensure False is returned if curl fails to download remote file
-
- Download remote file to a fake download directory. But, making
- something wrong to cause download_rpm returns false.
- """
- mock_get_download_dir.return_value = self.download_dir
-
- url = self.make_nonexistent_remote_file_url()
- ret = fedabipkgdiff_mod.download_rpm(url)
- self.assertFalse(ret)
-
-
class MockGlobalConfig(object):
"""Used to mock global_config
@@ -475,11 +417,15 @@ class MockGlobalConfig(object):
helpful for tests to contain all potential parsed (simulated)
options.
- Currently, only koji_server is required for running tests. If any new test
- cases need others, please add them add as class attribute directly.
+ Currently, only koji_server and dry_run are required for running
+ tests. If any new test cases need others, please add them add as
+ class attribute directly.
+
"""
koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
+ dry_run = False
+
def mock_get_session():
"""Used to mock get_session method to get mocked KojiSession instance"""
@@ -608,6 +554,64 @@ class GetPackageLatestBuildTest(unittest.TestCase):
session.get_package_latest_build, 'httpd', 'xxxx')
+class DownloadRPMTest(unittest.TestCase):
+ """Test case for download_rpm
+
+ Download a remote file, which is a local file simulating a remote file with
+ scheme file://, for example file:///tmp/a.txt, to download directory.
+ """
+
+ def setUp(self):
+ # Create a remote file for testing download of this file
+ self.fd, self.remote_filename = tempfile.mkstemp(
+ prefix=temp_file_or_dir_prefix)
+ # Whatever the content is, this case does not care about. Close it
+ # immediately.
+ os.close(self.fd)
+
+ # Used as a fake download directory to mock get_download_dir method
+ self.download_dir = tempfile.mkdtemp(prefix=temp_file_or_dir_prefix)
+
+ def tearDown(self):
+ os.remove(self.remote_filename)
+ shutil.rmtree(self.download_dir)
+
+ def make_remote_file_url(self):
+ """Make URL of remote file that is used for downloading this file"""
+ return 'file://{}'.format(self.remote_filename)
+
+ def make_nonexistent_remote_file_url(self):
+ """Return URL to a non-existent remote file"""
+ return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('__main__.fedabipkgdiff_mod.get_download_dir')
+ def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
+ """Enusre True is returned if curl succeeds to download remote file
+
+ Download remote file to a fake download directory. Ensure everything is
+ okay, and return value from download_rpm should be truth.
+ """
+ mock_get_download_dir.return_value = self.download_dir
+
+ url = self.make_remote_file_url()
+ ret = fedabipkgdiff_mod.download_rpm(url)
+ self.assertTrue(ret)
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('__main__.fedabipkgdiff_mod.get_download_dir')
+ def test_failed_to_download_a_rpm(self, mock_get_download_dir):
+ """Ensure False is returned if curl fails to download remote file
+
+ Download remote file to a fake download directory. But, making
+ something wrong to cause download_rpm returns false.
+ """
+ mock_get_download_dir.return_value = self.download_dir
+
+ url = self.make_nonexistent_remote_file_url()
+ ret = fedabipkgdiff_mod.download_rpm(url)
+ self.assertFalse(ret)
+
class BrewListRPMsTest(unittest.TestCase):
"""Test case for Brew.listRPMs"""
So I applied the resulting patch to
https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commit;h=57dcfb18f5599a5ec7d9faa8ad2ced33556ac4bd.
Thanks for your hard work!
Cheers,
--
Dodji
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Dodji Seketeli
@ 2016-01-01 0:00 ` Chenxiong Qi
0 siblings, 0 replies; 10+ messages in thread
From: Chenxiong Qi @ 2016-01-01 0:00 UTC (permalink / raw)
To: Dodji Seketeli, Chenxiong Qi; +Cc: libabigail
On 05/13/2016 06:57 AM, Dodji Seketeli wrote:
> Hello Chenxiong,
>
> [...]
>
>> This time, the patch is updated
>>
>> - more documentation inside fedabipkgdiff and runtestfedabipkgdiff.py.in
>> - using curl instead of wget to download packages from Koji
>> - a new manual (with Dodji's help, man page can be generated from this manual by sphnix)
>> - rebased, on latest commit fa5a5acbbcb1e
> Thank you for all these efforts!
>
> So I have reviewed this updated patch, made some changes and committed
> the result to the master branch of the git repository. Woohoo :-)
>
> Below are some comments about the parts of your patch where I made
> some changes. If you think we should amend them, please do not
> hesitate to say it.
>
> [...]
>
>> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
> [...]
>
>> +import xdg.BaseDirectory
> You are importing a new python module (xdg) that needs to be installed
> otherwise the execution of the fedabipkgdiff program just halts here.
> We thus need to ensure that the configure scripts detects when the
> python module is not installed, and gives a hint to the user saying
> that she needs to install it. So I added this hunk:
>
> --- a/configure.ac
> +++ b/configure.ac
> @@ -288,6 +288,7 @@ if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
> AX_PYTHON_MODULE(itertools, $FATAL, python2)
> AX_PYTHON_MODULE(shutil, $FATAL, python2)
> AX_PYTHON_MODULE(unittest, $FATAL, python2)
> + AX_PYTHON_MODULE(xdg, $FATAL, python2)
> AX_PYTHON_MODULE(koji, $FATAL, python2)
> AX_PYTHON_MODULE(mock, $FATAL, python2)
> ENABLE_FEDABIPKGDIFF=yes
>
> [...]
>
>> diff --git a/doc/manuals/Makefile.am b/doc/manuals/Makefile.am
>> index f23d1ab..5128bc0 100644
>> --- a/doc/manuals/Makefile.am
>> +++ b/doc/manuals/Makefile.am
>> @@ -10,7 +10,8 @@ conf.py \
>> index.rst \
>> libabigail-concepts.rst \
>> libabigail-overview.rst \
>> -libabigail-tools.rst
>> +libabigail-tools.rst \
>> +fedabipkgdiff
> Here there is no file named 'fedabipkgdiff' to be added to the set of
> files that are doing to be distributed in the tarball that is
> constructed by the 'make dist' command.
>
> What you want is to add the file fedabipkgdiff.rst. So I have added
> this the hunk below that fixes that.
>
> diff --git a/doc/manuals/Makefile.am b/doc/manuals/Makefile.am
> index 5128bc0..573fb3b 100644
> --- a/doc/manuals/Makefile.am
> +++ b/doc/manuals/Makefile.am
> @@ -11,7 +11,7 @@ index.rst \
> libabigail-concepts.rst \
> libabigail-overview.rst \
> libabigail-tools.rst \
> -fedabipkgdiff
> +fedabipkgdiff.rst
>
> Note that you could have detected the issue by typing the command:
>
> make distcheck
>
> which checks that constructing the tarball works correctly. That
> 'distcheck' target goes even further and tries to *compile* the
> tarball that was successfully built and runs all the tests. When
> "make distcheck" works, it means the software is ready to be released
> :-) We should always run make distcheck before submitting a patch, I
> believe.
>
> When looking at the tools/Makefile.am file I noticed that I forgot to
> add the fedabipkgdiff to the set of files that needs to be
> *distributed*. It was just meant to be installed in the /usr/bin/
> directory. I have thus added the hunk below to fix that:
>
> diff --git a/tools/Makefile.am b/tools/Makefile.am
> index 0d96215..3e53eb1 100644
> --- a/tools/Makefile.am
> +++ b/tools/Makefile.am
> @@ -7,7 +7,7 @@ else
> endif
>
> if ENABLE_FEDABIPKGDIFF
> - bin_SCRIPTS = fedabipkgdiff
> + dist_bin_SCRIPTS = fedabipkgdiff
> else
> noinst_SCRIPTS = fedabipkgdiff
> endif
>
> [...]
>
>> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
>
>> +HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home,
>> + os.path.splitext(os.path.basename(__file__))[0])
> As I said in my previous review, I prefer this to be XDG_CACHE_HOME,
> rather than XDG_DATA_HOME. The reason being that I consider the
> downloaded packages as being cached data, rather than application
> data. So I changed this to XDG_CACHE_HOME.
>
> The resulting hunk is:
>
> diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
> index e7c4785..f2b20be 100755
> --- a/tools/fedabipkgdiff
> +++ b/tools/fedabipkgdiff
> @@ -60,7 +60,7 @@ DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
> # The working directory where to hold all data including downloaded RPM
> # packages Currently, it's not configurable and hardcode here. In the future
> # version of fedabipkgdiff, I'll make it configurable by users.
> -HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home,
> +HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_cache_home,
> os.path.splitext(os.path.basename(__file__))[0])
>
> # Used to construct abipkgdiff command line argument, package and
> associated
>
> [...]
>
>> + def listBuilds(self, packageID, state=None, topone=None,
>> + selector=None, order_by=None, reverse=None):
> [...]
>
>> + builds = self.session.listBuilds(packageId=packageID, state=state)
> Here, packageId should be packageID instead. And this is causing a
> runtime error. So I fixed it. The resulting hunk is:
>
> @@ -417,7 +417,7 @@ class Brew(object):
> if order_by is not None and not isinstance(order_by, basestring):
> raise TypeError('order_by {0} is invalid.'.format(order_by))
>
> - builds = self.session.listBuilds(packageId=packageID, state=state)
> + builds = self.session.listBuilds(packageID=packageID, state=state)
> if selector is not None:
> builds = [build for build in builds if selector(build)]
> if order_by is not None:
>
> [...]
>
>> +def download_rpm(url):
> [...]
>
>> + cmd = 'curl --silent -C - -O {} > {}'.format(
>> + url, os.path.join(get_download_dir(),
>> + os.path.basename(url)))
> The resulting rpm file named
>
> os.path.join(get_download_dir(), os.path.basename(url))
>
> are empty here. So the comparison performed by abipkgdiff is failing
> on my system. This is because when you use the -O option, curl
> doesn't emit anything on standard output, at least on the curl 7.29.0
> version that I am using on my system.
>
> Also, I think the -C option is not critically useful here.
>
> Furthermore, I noticed that this program tries to download packages
> even when the --dry-run option is used. I think it's useful to avoid
> downloading packages when --dry-run is turned on.
>
> So I changed this using this hunk:
>
> @@ -621,9 +621,13 @@ def download_rpm(url):
> :return: True if a RPM is downloaded successfully, False otherwise.
> :rtype: bool
> """
> - cmd = 'curl --silent -C - -O {} > {}'.format(
> + cmd = 'curl --silent {} -o {}'.format(
> url, os.path.join(get_download_dir(),
> os.path.basename(url)))
> + if global_config.dry_run:
> + print 'DRY-RUN:', cmd
> + return
> +
> return_code = subprocess.call(cmd, shell=True)
> if return_code > 0:
> logger.error('curl fails with returned code: %d.', return_code)
>
> [...]
>
>> +def abipkgdiff(pkg_info1, pkg_info2):
> [...]
>
>> + cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
>> + pkg_info1.debuginfo_package.downloaded_file,
>> + pkg_info2.debuginfo_package.downloaded_file,
>> + pkg_info1.package.downloaded_file,
>> + pkg_info2.package.downloaded_file)
> So here fedabipkgdiff is comparing the ABI of *all* binaries in the
> packages, including executable binaries. By default I think it makes
> more sense to only compare the ABI of shared libraries, not executable
> binaries. And it's faster too. So I changed this by doing:
>
> @@ -672,7 +672,7 @@ def abipkgdiff(pkg_info1, pkg_info2):
> :return: return code of underlying abipkgdiff execution.
> :rtype: int
> """
> - cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
> + cmd = 'abipkgdiff --dso-only --d1 {0} --d2 {1} {2} {3}'.format(
> pkg_info1.debuginfo_package.downloaded_file,
> pkg_info2.debuginfo_package.downloaded_file,
> pkg_info1.package.downloaded_file,
>
> I made wording changes in the build_commandline_args_parser function.
> The resulting hunk is:
>
> @@ -924,7 +928,7 @@ def diff_two_nvras_from_koji():
>
> def build_commandline_args_parser():
> parser = argparse.ArgumentParser(
> - description='Run abipkgdiff against RPM packages from koji')
> + description='Compare ABI of shared libraries in RPM packages from the Koji build system')
>
> parser.add_argument(
> 'NVR',
> @@ -936,20 +940,20 @@ def build_commandline_args_parser():
> required=False,
> dest='dry_run',
> action='store_true',
> - help='Don\'t actually run abipkgdiff. The commands that should be '
> + help='Don\'t actually do the work. The commands that should be '
> 'run will be sent to stdout.')
> parser.add_argument(
> '--from',
> required=False,
> metavar='DISTRO',
> dest='from_distro',
> - help='baseline Fedora distro, for example, fc23')
> + help='baseline Fedora distribution name, for example, fc23')
> parser.add_argument(
> '--to',
> required=False,
> metavar='DISTRO',
> dest='to_distro',
> - help='which Fedora distro to compare, for example, fc24')
> + help='Fedora distribution name to compare against the baseline, for example, fc24')
> parser.add_argument(
> '-a',
> '--all-subpackages',
>
> After I started to use the global_config.dry_run member variable in
> download_rpm, I had to update the unit tests that exercise that member
> function in tests/runtestfedabipkgdiff.py.in.
>
> To do so, I had to move the test class DownloadRPMTest to *after* the
> class MockGlobalConfig is defined, so that member functions of
> DownloadRPMTest can use the MockGlobalConfig, which is now needed by
> the member functions of DownloadRPMTest that exercise the
> download_rpm function. Of course, I had to add a dry_run data member
> to MockGlobalConfig and update the comment in there accordingly.
>
> The resulting hunk is:
>
> diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
> index 84a4320..483cfdb 100755
> --- a/tests/runtestfedabipkgdiff.py.in
> +++ b/tests/runtestfedabipkgdiff.py.in
> @@ -410,64 +410,6 @@ class RunAbipkgdiffTest(unittest.TestCase):
> self.pkg2_infos)
> self.assertTrue(result > 0)
>
> -
> -class DownloadRPMTest(unittest.TestCase):
> - """Test case for download_rpm
> -
> - Download a remote file, which is a local file simulating a remote file with
> - scheme file://, for example file:///tmp/a.txt, to download directory.
> - """
> -
> - def setUp(self):
> - # Create a remote file for testing download of this file
> - self.fd, self.remote_filename = tempfile.mkstemp(
> - prefix=temp_file_or_dir_prefix)
> - # Whatever the content is, this case does not care about. Close it
> - # immediately.
> - os.close(self.fd)
> -
> - # Used as a fake download directory to mock get_download_dir method
> - self.download_dir = tempfile.mkdtemp(prefix=temp_file_or_dir_prefix)
> -
> - def tearDown(self):
> - os.remove(self.remote_filename)
> - shutil.rmtree(self.download_dir)
> -
> - def make_remote_file_url(self):
> - """Make URL of remote file that is used for downloading this file"""
> - return 'file://{}'.format(self.remote_filename)
> -
> - def make_nonexistent_remote_file_url(self):
> - """Return URL to a non-existent remote file"""
> - return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
> -
> - @patch('__main__.fedabipkgdiff_mod.get_download_dir')
> - def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
> - """Enusre True is returned if curl succeeds to download remote file
> -
> - Download remote file to a fake download directory. Ensure everything is
> - okay, and return value from download_rpm should be truth.
> - """
> - mock_get_download_dir.return_value = self.download_dir
> -
> - url = self.make_remote_file_url()
> - ret = fedabipkgdiff_mod.download_rpm(url)
> - self.assertTrue(ret)
> -
> - @patch('__main__.fedabipkgdiff_mod.get_download_dir')
> - def test_failed_to_download_a_rpm(self, mock_get_download_dir):
> - """Ensure False is returned if curl fails to download remote file
> -
> - Download remote file to a fake download directory. But, making
> - something wrong to cause download_rpm returns false.
> - """
> - mock_get_download_dir.return_value = self.download_dir
> -
> - url = self.make_nonexistent_remote_file_url()
> - ret = fedabipkgdiff_mod.download_rpm(url)
> - self.assertFalse(ret)
> -
> -
> class MockGlobalConfig(object):
> """Used to mock global_config
>
> @@ -475,11 +417,15 @@ class MockGlobalConfig(object):
> helpful for tests to contain all potential parsed (simulated)
> options.
>
> - Currently, only koji_server is required for running tests. If any new test
> - cases need others, please add them add as class attribute directly.
> + Currently, only koji_server and dry_run are required for running
> + tests. If any new test cases need others, please add them add as
> + class attribute directly.
> +
> """
> koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
>
> + dry_run = False
> +
>
> def mock_get_session():
> """Used to mock get_session method to get mocked KojiSession instance"""
> @@ -608,6 +554,64 @@ class GetPackageLatestBuildTest(unittest.TestCase):
> session.get_package_latest_build, 'httpd', 'xxxx')
>
>
> +class DownloadRPMTest(unittest.TestCase):
> + """Test case for download_rpm
> +
> + Download a remote file, which is a local file simulating a remote file with
> + scheme file://, for example file:///tmp/a.txt, to download directory.
> + """
> +
> + def setUp(self):
> + # Create a remote file for testing download of this file
> + self.fd, self.remote_filename = tempfile.mkstemp(
> + prefix=temp_file_or_dir_prefix)
> + # Whatever the content is, this case does not care about. Close it
> + # immediately.
> + os.close(self.fd)
> +
> + # Used as a fake download directory to mock get_download_dir method
> + self.download_dir = tempfile.mkdtemp(prefix=temp_file_or_dir_prefix)
> +
> + def tearDown(self):
> + os.remove(self.remote_filename)
> + shutil.rmtree(self.download_dir)
> +
> + def make_remote_file_url(self):
> + """Make URL of remote file that is used for downloading this file"""
> + return 'file://{}'.format(self.remote_filename)
> +
> + def make_nonexistent_remote_file_url(self):
> + """Return URL to a non-existent remote file"""
> + return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
> +
> + @patch('fedabidiff.global_config', new=MockGlobalConfig)
> + @patch('__main__.fedabipkgdiff_mod.get_download_dir')
> + def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
> + """Enusre True is returned if curl succeeds to download remote file
> +
> + Download remote file to a fake download directory. Ensure everything is
> + okay, and return value from download_rpm should be truth.
> + """
> + mock_get_download_dir.return_value = self.download_dir
> +
> + url = self.make_remote_file_url()
> + ret = fedabipkgdiff_mod.download_rpm(url)
> + self.assertTrue(ret)
> +
> + @patch('fedabidiff.global_config', new=MockGlobalConfig)
> + @patch('__main__.fedabipkgdiff_mod.get_download_dir')
> + def test_failed_to_download_a_rpm(self, mock_get_download_dir):
> + """Ensure False is returned if curl fails to download remote file
> +
> + Download remote file to a fake download directory. But, making
> + something wrong to cause download_rpm returns false.
> + """
> + mock_get_download_dir.return_value = self.download_dir
> +
> + url = self.make_nonexistent_remote_file_url()
> + ret = fedabipkgdiff_mod.download_rpm(url)
> + self.assertFalse(ret)
> +
> class BrewListRPMsTest(unittest.TestCase):
> """Test case for Brew.listRPMs"""
>
> So I applied the resulting patch to
> https://sourceware.org/git/gitweb.cgi?p=libabigail.git;a=commit;h=57dcfb18f5599a5ec7d9faa8ad2ced33556ac4bd.
>
> Thanks for your hard work!
>
> Cheers,
>
Thank you very much for your great review. I'm excited to see this is
merged and will help Fedora packagers to do their great contributions. :)
--
Regards,
Chenxiong Qi
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: a new tool fedabipkgdiff
2016-01-01 0:00 ` Dodji Seketeli
@ 2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
0 siblings, 1 reply; 10+ messages in thread
From: Chenxiong Qi @ 2016-01-01 0:00 UTC (permalink / raw)
To: Dodji Seketeli, Chenxiong Qi; +Cc: libabigail
[-- Attachment #1: Type: text/plain, Size: 25242 bytes --]
> On Thursday, March 31, 2016 7:03 PM, Dodji Seketeli <dodji@redhat.com> wrote:
> > Hello Chenxiong,
>
> Sorry for my being late to review this. It just felt through the cracks
> while I was looking as something else. Oh well.
>
> So I reviewed it and I must say this is good stuff :-)
>
> I looked at it in details and the comments I have now are mostly about
> documentations and the like.
>
> Please find my comments below.
>
>> * autoconf-archive/ax_compare_version.m4: New file copied from the
>
> Each of these lines should start with a 'tab' character (you know, the
> \t character), as specified in the file COMMIT-LOG-GUIDELINES:
>
> The subsequent lines should have the form of the Body of a GNU ChangeLog
> entry, i.e:
>
> * file1.c (func1): Changed foo in this function.
> (func2): Changed blah in that function
> * file2.c (func_foo): Changed something here.
>
> Note that before the '*', there is a tab that is 8 spaces long.
> Also
> note that right after the '*', there is a space.
>
> To ease the task, I have edited your ChangeLog. Here is the result,
> that you can add to your patch:
>
> * autoconf-archive/ax_compare_version.m4: New file copied from the
> autoconf-archive project.
> * autoconf-archive/ax_prog_python_version.m4: Likewise.
> * autoconf-archive/ax_python_module.m4: Likewise.
> * Makefile.am: Add the new files above to the source distribution.
> * configure.ac: Include the new m4 macros from the autoconf
> archive. Add a new --enable-fedabipkgdiff option. Update the
> report at the end of the configure process to show the status of
> the fedabipkgdiff feature. Add check for prerequisite python
> modules itertools, shutil, unittest and mock. These are necessary
> for the unit test of fedabipkgdiff. Generate
> tests/runtestfedabipkgdiff.py into the build directory, from the
> tests/runtestfedabipkgdiff.py.in input file.
> * tools/Makefile.am: Include the fedabipkgdiff to the source
> distribution and install it if the "fedabipkgdiff" feature is
> enabled.
> * tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
> runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
> autoconf template file in here.
> * tests/runtestfedabipkgdiff.py.in: New unit test file.
> * tools/fedabipkgdiff: New tool fedabipkgdiff.
> [...]
>
>> diff --git a/tests/runtestfedabipkgdiff.py.in
> b/tests/runtestfedabipkgdiff.py.in
>> new file mode 100755
>> index 0000000..3087212
>> --- /dev/null
>> +++ b/tests/runtestfedabipkgdiff.py.in
>> @@ -0,0 +1,450 @@
>> +#!/usr/bin/python
>> +# -*- coding: utf-8 -*-
>> +# -*- Mode: Python
>> +#
>> +# This file is part of the GNU Application Binary Interface Generic
>> +# Analysis and Instrumentation Library. This program is free
>> +# software; you can redistribute it and/or modify it under the terms
>> +# of the GNU General Public License as published by the Free Software
>> +# Foundation; either version 3, or (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful, but
>> +# WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> +# General Lesser Public License for more details.
>> +#
>> +# You should have received a copy of the GNU Lesser General Public
>> +# License along with this program; see the file COPYING-LGPLV3. If
>> +# not, see <http://www.gnu.org/licenses/>.
>
> This is a program, not a library. In the project, by default,
> programs are GPL. It's only the library that is LGPL. So if this is
> an oversight, I guess we should fix it and change this license to GPL.
>
>> +#
>> +# Author: Chenxiong Qi
>> +
>> +import os
>> +import itertools
>> +import shutil
>> +import unittest
>> +
>> +import koji
>
> In general, I find that this file lacks comments. There should be a
> general comment at the beginning of the file that introduces the kind
> of tests that are performed here. For instance,
> tests/test-diff-pkg.cc has:
>
> /// @file
> ///
> /// This test harness program computes the ABI changes between ELF
> /// binaries present inside input packages. Some of the input
> /// packages have debuginfo, some don't. The resulting ABI change
> /// report is then compared with a reference one.
> ///
> /// The set of input files and reference reports to consider should be
> /// present in the source distribution, which means they must be
> /// referenced in tests/data/Makefile.am by the EXTRA_DIST variable.
>
> And if you look into that file, there are various other comments that
> describe how the test is organized etc. I think we should try and do
> the same for all the tests that we add.
>
> For instance:
>
> [...]
>
>> +counter = itertools.count(0)
>
> What is this global variable for? We should have a comment for this.
>
>
>> +class UtilsTest(unittest.TestCase):
>
> We should have an introductory comment for this, I believe.
>
>> +
>> + def test_is_fedora_distro(self):
>
> Same for this test. What is the philosophy behind this? What is the
> kind of "distro strings" we want to accept? etc.
>
>> + distro = 'fc5'
>> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'f5'
>> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'fc23'
>> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'fc'
>> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'fc234'
>> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'el7'
>> + self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> + distro = 'el7_2'
>> + self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
>> +
>> +
>> +class RPMTest(unittest.TestCase):
>> + """Test case for RPM class"""
>
> For instance, I would have expected this comment to say something
> introductory about the tests performed on the fedabipkgdiff.RPM
> class.
>
>> +
>> + def setUp(self):
>
> Here, I would have expected a comment about what this
> debuginfo_rpm_info data member is, saying that it must be the same
> type as the parameter type expected by the constructor of the
> fedabipkgdiff.RPM class. So if that constructor's parameter type
> changes, this test must be updated. Something like that.
>
>> + self.debuginfo_rpm_info = {
>> + 'arch': 'i686',
>> + 'name': 'httpd-debuginfo',
>> + 'release': '1.fc22',
>> + 'version': '2.4.18'
>> + }
>
> Likewise.
>
>> + self.rpm_info = {
>> + 'arch': 'x86_64',
>> + 'name': 'httpd',
>> + 'release': '1.fc22',
>> + 'version': '2.4.18'
>> + }
>> +
>> + def test_attribute_access(self):
>
> Please add a comment saying what this test does. At least its
> intent. For instance:
>
> This test enforces the exported interface of the fedabipkgdiff.RPM
> class. If a new data member is added to that type, removed or
> changed in that class, please update this test accordingly.
>
> Maybe repeat here that the data members of the RPM class are the same
> as the data members of the poorly documented return type of the koji
> getRPM request, and so whenever that return type changes, we need to
> update this test. You know, something like that. Otherwise, there is
> too much implicit knowledge here, and this makes maintenance painful
> for anyone would would have to take over tomorrow. And this is
> especially true for a program in python where types are so loosely
> defined and thus where type mis-matches won't be caught until the
> program's runtime.
>
> etc.
>
>> + rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
>> + self.assertEquals(self.debuginfo_rpm_info['arch'],
> rpm.arch)
>> + self.assertEquals(self.debuginfo_rpm_info['name'],
> rpm.name)
>> + self.assertEquals(self.debuginfo_rpm_info['release'],
> rpm.release)
>> + self.assertEquals(self.debuginfo_rpm_info['version'],
> rpm.version)
>
> [...]
>
> I think all test cases should be documented similarly. And also, all
> the classes that are helper classes for the tests should be
> documented.
>
>
>> --- /dev/null
>> +++ b/tools/fedabipkgdiff
>> @@ -0,0 +1,805 @@
>> +#!/usr/bin/env python
>> +# -*- coding: utf-8 -*-
>> +# -*- Mode: Python
>> +#
>> +# This file is part of the GNU Application Binary Interface Generic
>> +# Analysis and Instrumentation Library. This program is free
>> +# software; you can redistribute it and/or modify it under the terms
>> +# of the GNU General Public License as published by the Free Software
>> +# Foundation; either version 3, or (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful, but
>> +# WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
>> +# General Lesser Public License for more details.
>> +#
>> +# You should have received a copy of the GNU Lesser General Public
>> +# License along with this program; see the file COPYING-LGPLV3. If
>> +# not, see <http://www.gnu.org/licenses/>.
>
> Likewise, I believe this should be GPL.
>
>> +#
>> +# Author: Chenxiong Qi
>> +
>
> Please add an introductory comment here, explaining the intend of the
> tool. Just like what is done for in tools/abidw.cc:
>
> /// @file
> ///
> /// This program reads an elf file, try to load its debug info (in
> /// DWARF format) and emit it back in a set of "text sections" in
> native
> /// libabigail XML format.
>
>
> [...]
>
> Generally speaking, I would like to see all the function parameter
> *types* and return values to be documented.
>
> [...]
>
>> +def log_call(func):
>> + def proxy(*args, **kwargs):
>
> Please add a comment that says that this is a decorator. And please
> tell what we expect *args and **kwargs to be. (the arguments of func,
> as well as the key word arguments, if any).
>
> [...]
>
>
>> +class RPM(object):
>> + """Represeting a RPM"""
>> +
>> + def __init__(self, data):
>> + """Initialize a RPM object
>> +
>> + :param dict data: a dict representing a RPM information got from
> koji
>> + API, either listRPMs or getRPM
>
> What are the keys and values expected in that dict? Granted the koji
> API is so "lightly" documented, it doesn't really help. But I
> think
> we could write an example of the dict keys/values that *we* expect
> here, to ease maintenance.
>
> Also, you could say something about the fact that the set of data
> members expected by this class is enforced in the unit test case
>
> [...]
>
>> + @property
>> + def nvra(self):
>> + return '%(name)s-%(version)s-%(release)s.%(arch)s' %
> self.data
>> +
>> + @property
>> + def filename(self):
>> + return '{0}.rpm'.format(self.nvra)
>> +
>> + @property
>> + def is_debuginfo(self):
>> + """Check if a RPM is a debuginfo"""
>
> I would have said:
>
> "Check if the name of the current RPM denotes a debug info
> package"
>
>> + return koji.is_debuginfo(self.data['name'])
>> +
>> + @property
>> + def download_url(self):
>> + """Get the URL from where to download from
> koji"""
>> + build = session.getBuild(self.build_id)
>> + return os.path.join(pathinfo.build(build),
> pathinfo.rpm(self.data))
>> +
>> + @property
>> + def downloaded_file(self):
>> + """Get a pridictable downloaded file name with
> absolute path"""
>> + # arch should be removed from the result returned from
> PathInfo.rpm
>> + filename = os.path.basename(pathinfo.rpm(self.data))
>> + return os.path.join(get_download_dir(), filename)
>> +
>> + @property
>> + def is_downloaded(self):
>
> Please add a comment here.
>
>> + return os.path.exists(self.downloaded_file)
>> +
>> +
>> +class LocalRPM(RPM):
>> + """Representing a local RPM
>> +
>> + Local RPM means the one that could be already downloaded or built from
>> + where I can find it
>> + """
>> +
>> + def __init__(self, filename):
>
> Please add a comment here. What is the type of the "filename"
> parameter etc...
>
>> + self.local_filename = filename
>> + self.data = koji.parse_NVRA(os.path.basename(filename))
>> +
>> + @property
>> + def downloaded_file(self):
>
> Comments.
>
>> +
>> +class Brew(object):
>> + """Proxy to kojihub XMLRPC with additional extensions
> to fedabipkgdiff
>> +
>> + kojihub XMLRPC APIs are well-documented in koji's source code. For
> more
>> + details information, please refer to class RootExports within
> kojihub.py.
>> + """
>
> Please add an html link to the koji source code where the XMLRPC APIs
> are documented.
>
>> +
>> + def __init__(self, baseurl):
>> + """Initialize Brew that is a proxy to
> koji.ClientSession"""
>> + self.session = koji.ClientSession(baseurl)
>> +
>> + @log_call
>> + def listRPMs(self, selector=None, **kwargs):
>> + """Proxy to kojihub.listRPMs
>> +
>> + :param selector: to adjust if a RPM should be selected
>> + :type selector: a callable object
>> + :param kwargs: keyword parameters accepted by kojihub.listRPMs
>
> Here if you could give examples of the keywords (and their arguments)
> that we'd pass, it would be good. I know the reader can always go dig
> into kojihub.listRPMs, but examples would help. I think redundancy is
> good here.
>
>> + :type kwargs: dict
>> + :return: a list of RPMs, each of them is a dict object
>
> Again, an example of the dict object that we expect would be good.
>
>> + :rtype: list
>> + """
>> + if selector:
>> + assert hasattr(selector, '__call__'), 'selector
> should be callable'
>> + rpms = self.session.listRPMs(**kwargs)
>> + if selector:
>> + rpms = [rpm for rpm in rpms if selector(rpm)]
>> + return rpms
>> +
>> + @log_call
>> + def getRPM(self, *args, **kwargs):
>
> Comments.
>
>> + rpm = self.session.getRPM(*args, **kwargs)
>> + if rpm is None:
>> + raise RpmNotFound('Cannot find RPM
> {0}'.format(args[0]))
>> + return rpm
>> +
>> + @log_call
>> + def listBuilds(self, topone=None, selector=None, order_by=None,
>> + reverse=None, **kwargs):
>> + """Proxy to kojihub.listBuilds to list completed
> builds
>
> Please document here (yeah I know it's redundant) the parameters that
> kojihub.listBuilds expects.
>
>> +
>> + Suport additional two keyword parameters:
>> +
>> + :param bool topone: whether to return the top first one
>> + :param selector: a callable object used to select specific subset
> of
>> + builds
>> + :type selector: callable object
>> + :param str order_by: the attribute name by which to order the
> builds,
>> + for example, name, version, or nvr.
>> + :param bool reverse: whether to order builds reversely
>> + :param dict kwargs: keyword parameters accepted by
> kojihub.listBuilds
>> + :return: a list of builds, even if just return only one build
>> + :rtype: list
>> + """
>> + if 'state' not in kwargs:
>> + kwargs['state'] =
> koji.BUILD_STATES['COMPLETE']
>> +
>> + if selector is not None and not hasattr(selector,
> '__call__'):
>> + raise TypeError(
>> + '{0} is not a callable
> object.'.format(str(selector)))
>> +
>> + if order_by is not None and not isinstance(order_by, basestring):
>> + raise TypeError('order_by {0} is
> invalid.'.format(order_by))
>> +
>> + builds = self.session.listBuilds(**kwargs)
>> + if selector is not None:
>> + builds = [build for build in builds if selector(build)]
>> + if order_by is not None:
>> + # FIXME: is it possible to sort builds by using opts parameter
> of
>> + # listBuilds
>> + builds = sorted(builds,
>> + key=lambda item: item[order_by],
>> + reverse=reverse)
>> + if topone:
>> + builds = builds[0:1]
>> +
>> + return builds
>> +
>> + @log_call
>> + def getPackage(self, name):
>> + """Proxy to kojihub.getPackage
>> +
>> + :param str name: package name
>
> Please give example of the package name that we expect, so that a user
> of this function has an idea of the correct form expected.
>
>> + :return: a dict object representing a package
>> + :rtype: dict
>> + """
>> + package = self.session.getPackage(name)
>> + if package is None:
>> + package = self.session.getPackage(name.rsplit('-',
> 1)[0])
>> + if package is None:
>> + raise KojiPackageNotFound(
>> + 'Cannot find package {0}.'.format(name))
>> + return package
>> +
>> + @log_call
>> + def getBuild(self, *args, **kwargs):
>> + """Proxy to kojihub.getBuild"""
>
> Please describe the (keyword) arguments expected.
>
>> + return self.session.getBuild(*args, **kwargs)
>> +
>
>> + @log_call
>> + def get_rpm_build_id(self, name, version, release, arch=None):
>> + """Get build ID that contains a rpm with specific
> nvra
>> +
>> + If arch is omitted, a rpm can be identified easily by its N-V-R-A.
>> +
>> + If arch is omitted, name is used to get associated package, and
> then
>> + to get the build.
>> +
>> + :param str name: name of a rpm
>> + :param str version: version of a rpm
>> + :param str release: release of a rpm
>
> I think that here, example of what you expect as release (at least)
> would be interesting.
>
> [...]
>
>> +
>> + @log_call
>> + def get_package_latest_build(self, package_name, distro):
>> + """Get latest build from a package
>> +
>> + :param str package_name: from which package to get the latest
> build
>> + :param str distro: which distro the latest build belongs to
>
> The form of what is expected as distro would be important too. That
> would help to maintain the test suite.
>
> [...]
>
> +@log_call
> +def get_session():
>
> Comments.
>
> + return Brew(global_config.koji_server)
>
> [...]
>
> +def get_download_dir():
> + """Return the directory holding all downloaded
> rpms"""
> + download_dir = os.path.join(HOME_DIR, 'downloads')
>
> I would use the standard $XDG_CACHE_HOME place, to store the
> downloaded packages:
> https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html.
> That is, ~/.local/fedabipkgdiff.
>
> I could also use a fedabipkgdiff --clean-cache option to clean the
> cache cache, but we can add that this later, after the patch is in.
>
> + if not os.path.exists(download_dir):
> + os.makedirs(download_dir)
> + return download_dir
> +
>
>> +@log_call
>> +def abipkgdiff(pkg_info1, pkg_info2):
>> + """Run abipkgdiff against found two RPM
> packages"""
>
> Please document the expected types of the parameters here.
>
> + print 'ABI check on {0} and
> {1}'.format(pkg_info1.package.filename,
> + pkg_info2.package.filename)
>
> Please, rather print:
>
> 'Comparing the ABI of binaries between {0} and {1}:'
>
> [...]
>
>> +def magic_construct(rpms):
>> + """Construct RPMs into a magic structure
>> +
>> + Convert list of
>> +
>> + foo-1.0-1.fc22.i686
>> + foo-debuginfo-1.0-1.fc22.i686
>> + foo-devel-1.0-1.fc22.i686
>> +
>> + to list of
>> +
>> + (foo-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
>> + (foo-devel-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
>> + """
>> + debuginfo = None
>> + packages = []
>> + for rpm in rpms:
>> + if rpm.is_debuginfo:
>> + debuginfo = rpm
>> + else:
>> + packages.append(rpm)
>> + return [PkgInfo(package, debuginfo) for package in packages]
>> +
>> +
>> +@log_call
>> +def run_abipkgdiff(pkg1_infos, pkg2_infos):
>> + """Run abipkgdiff
>> +
>> + If one of the executions finds ABI differences, the return code is the
>> + return code from abipkgdiff.
>> +
>> + :param dict pkg1_infos: a dict mapping from arch to list of rpms, that
> is
>> + returned from method make_rpms_usable_for_abipkgdiff
>
> Rather than refer to what make_rpms_usable_for_abipkgdiff returns,
> please redundantly describe the dict here, giving a real example, so
> that a user willing to use this function can see what she should feed
> it with.
>
> [...]
>
>> +@log_call
>> +def diff_local_rpm_with_latest_rpm_from_koji():
>> + """Diff against local rpm and remove latest rpm
>> +
>> + This operation handles a local rpm and debuginfo rpm and remote ones
>> + located in remote Koji server, that has specific distro specificed by
>> + argument --from.
>> +
>> + 1/ Suppose the packager has just locally built a package named
>> + foo-3.0.fc24.rpm. To compare the ABI of this locally build package
> with the
>> + latest stable package from Fedora 23, one would do:
>> +
>> + fedabipkgdiff --from f23 ./foo-3.0.fc24.rpm
>> + """
>
> Great comment, thanks.
>
> [...]
>
>
>> +@log_call
>> +def make_rpms_usable_for_abipkgdiff(rpms):
>> + """
>> + Construct result that contains mappings from arch to download url and
>> + downloaded rpm filename of rpm and debuginfo rpm
>> +
>> + :return: a mapping from an arch to a list of rpms
>
> Please give an example of this dict.
>
> [...]
>
>
>> +@log_call
>> +def diff_rpms_with_nvra(name, version, release, arch=None,
>> + all_subpackages=None):
>
> Comments.
>
>> + build_id = session.get_rpm_build_id(name, version, release, arch)
>> + rpms = session.select_rpms_from_a_build(build_id, name, arches=arch,
>> +
> select_subpackages=all_subpackages)
>> + return make_rpms_usable_for_abipkgdiff(rpms)
>> +
>
> Also, I wouldn't call this function "diff_rpms_with_nvra", because
> the
> function doesn't perform any diffing, does it? Rather, my
> understanding is that it constructs the two sets of package
> descriptors that would be passed to abipkgdiff.
>
>
>> +@log_call
>> +def diff_two_nvras_from_koji():
>> + """Diff two nvras from koji
>> +
>> + The arch probably omits, that means febabipkgdiff will diff all
> arches. If
>
> I would say "the arch is probably omitted ..."
>
> [...]
>
> +def main():
>
> [...]
>
>
> + else:
> + print >>sys.stderr, 'Unknown arguments. Please refer to
> -h.'
> + returncode = 1
>
> Please, refer to --help, rather -h just like what is done in the other
> libabigail tools.
>
> +
> + return returncode
>
> All in all, I am wondering if we shouldn't add some progress
> indication to the user, just like what "dnf" does. You know, saying
> things like "we are downloading this package etc, and now we are
> comparing these packages, etc." Otherwise, the time taken can seem
> quite long. But we can add this later, after the patch has been
> committed, I guess.
Maybe, a simple way is to use --verbose option to print such information next step.
>
> Also, we should arrange for the downloads to happen in parallel,
> somehow. Again, this can happen after the patch is in.
Haha, I know what you mean. Yeah, it's worth to do this.
>
> Now, the other big thing missing from this patch is a manual.
>
> That is really important. Every single libabigail tool has a
> documentation in docs/manuals. Documentation is in the restructured
> text format, and we use python-sphinx to generate html, man and info
> variants for it. The manual should also say where the packages are
> downloaded.
>
> If you don't feel like adding a manual for this tool, please tell me,
> I'll help.
>
> I love this patch very much! Thank you for your hard work on this.
> This is simply awesome. I cannot wait for it to get in :-)
>
> Cheers,
>
> --
> Dodji
>
This time, the patch is updated
- more documentation inside fedabipkgdiff and runtestfedabipkgdiff.py.in
- using curl instead of wget to download packages from Koji
- a new manual (with Dodji's help, man page can be generated from this manual by sphnix)
- rebased, on latest commit fa5a5acbbcb1e
Hello, Mr. Dodji, please review. Thanks.
Regards,
Chenxiong Qi
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-new-tool-of-fedabipkgdiff.patch --]
[-- Type: text/x-patch, Size: 87961 bytes --]
From ae5c2c6704c764946e7dca748a20eda378e15f56 Mon Sep 17 00:00:00 2001
From: Chenxiong Qi <cqi@redhat.com>
Date: Tue, 9 Feb 2016 18:05:33 +0800
Subject: [PATCH] new tool of fedabipkgdiff
fedabipkgdiff is a convenient way for Fedora packagers to inspect ABI
compatibility issues quickly.
Currently with the first version of fedabipkgdiff, you can invoke it in
following ways.
fedabipkgdiff --from fc23 foo-0.1-1.fc23.x86_64.rpm
fedabipkgdiff --from fc23 --to fc24 foo
fedabipkgdiff foo-0.1-1.fc23 foo-0.1-1.fc24
fedabipkgdiff foo-0.1-1.fc23.i686 foo-0.1-1.fc24.i686
fedabipkgdiff --all-subpackages foo-0.1-1.fc23 foo-0.1-1.fc24
* autoconf-archive/ax_compare_version.m4: New file copied from the
autoconf-archive project.
* autoconf-archive/ax_prog_python_version.m4: Likewise.
* autoconf-archive/ax_python_module.m4: Likewise.
* Makefile.am: Add the new files above to the source distribution.
* configure.ac: Include the new m4 macros from the autoconf
archive. Add a new --enable-fedabipkgdiff option. Update the report
at the end of the configure process to show the status of the
fedabipkgdiff feature. Add check for prerequisite python modules
itertools, shutil, unittest and mock. These are necessary for the
unit test of fedabipkgdiff. Generate tests/runtestfedabipkgdiff.py
into the build directory, from the tests/runtestfedabipkgdiff.py.in
input file.
* tools/Makefile.am: Include the fedabipkgdiff to the source
distribution and install it if the "fedabipkgdiff" feature is
enabled.
* tests/Makefile.am: Rename runtestfedabipkgdiff.sh into
runtestfedabipkgdiff.py. Add the new runtestfedabipkgdiff.py.in
autoconf template file in here.
* tests/runtestfedabipkgdiff.py.in: New unit test file.
* tools/fedabipkgdiff: New tool fedabipkgdiff.
* doc/manuals/fedabipkgdiff.rst: New manual.
Signed-off-by: Chenxiong Qi <cqi@redhat.com>
---
.gitignore | 6 +
Makefile.am | 3 +
autoconf-archive/ax_compare_version.m4 | 177 +++++
autoconf-archive/ax_prog_python_version.m4 | 66 ++
autoconf-archive/ax_python_module.m4 | 56 ++
configure.ac | 86 ++-
doc/manuals/Makefile.am | 3 +-
doc/manuals/conf.py | 1 +
doc/manuals/fedabipkgdiff.rst | 108 +++
doc/manuals/libabigail-tools.rst | 1 +
tests/Makefile.am | 6 +-
tests/runtestfedabipkgdiff.py.in | 634 +++++++++++++++++
tools/Makefile.am | 6 +
tools/fedabipkgdiff | 1050 ++++++++++++++++++++++++++++
14 files changed, 2200 insertions(+), 3 deletions(-)
create mode 100644 autoconf-archive/ax_compare_version.m4
create mode 100644 autoconf-archive/ax_prog_python_version.m4
create mode 100644 autoconf-archive/ax_python_module.m4
create mode 100644 doc/manuals/fedabipkgdiff.rst
create mode 100755 tests/runtestfedabipkgdiff.py.in
create mode 100755 tools/fedabipkgdiff
diff --git a/.gitignore b/.gitignore
index bb7c42a..a60cadb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ Makefile.in
*.lo
*.o
*~
+*.swp
/aclocal.m4
/autom4te.cache/
@@ -17,3 +18,8 @@ Makefile.in
/include/abg-version.h
/*.pc
+
+.tags
+build/
+TAGS
+fedabipkgdiffc
\ No newline at end of file
diff --git a/Makefile.am b/Makefile.am
index cdee0db..29bed61 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -12,6 +12,9 @@ pkgconfig_DATA = libabigail.pc
#dist_bashcompletion_DATA =
EXTRA_DIST = \
+autoconf-archive/ax_python_module.m4 \
+autoconf-archive/ax_prog_python_version.m4 \
+autoconf-archive/ax_compare_version.m4 \
NEWS README COPYING ChangeLog \
COPYING-LGPLV2 COPYING-LGPLV3 \
COPYING-GPLV3 gen-changelog.py \
diff --git a/autoconf-archive/ax_compare_version.m4 b/autoconf-archive/ax_compare_version.m4
new file mode 100644
index 0000000..74dc0fd
--- /dev/null
+++ b/autoconf-archive/ax_compare_version.m4
@@ -0,0 +1,177 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_compare_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_COMPARE_VERSION(VERSION_A, OP, VERSION_B, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# This macro compares two version strings. Due to the various number of
+# minor-version numbers that can exist, and the fact that string
+# comparisons are not compatible with numeric comparisons, this is not
+# necessarily trivial to do in a autoconf script. This macro makes doing
+# these comparisons easy.
+#
+# The six basic comparisons are available, as well as checking equality
+# limited to a certain number of minor-version levels.
+#
+# The operator OP determines what type of comparison to do, and can be one
+# of:
+#
+# eq - equal (test A == B)
+# ne - not equal (test A != B)
+# le - less than or equal (test A <= B)
+# ge - greater than or equal (test A >= B)
+# lt - less than (test A < B)
+# gt - greater than (test A > B)
+#
+# Additionally, the eq and ne operator can have a number after it to limit
+# the test to that number of minor versions.
+#
+# eq0 - equal up to the length of the shorter version
+# ne0 - not equal up to the length of the shorter version
+# eqN - equal up to N sub-version levels
+# neN - not equal up to N sub-version levels
+#
+# When the condition is true, shell commands ACTION-IF-TRUE are run,
+# otherwise shell commands ACTION-IF-FALSE are run. The environment
+# variable 'ax_compare_version' is always set to either 'true' or 'false'
+# as well.
+#
+# Examples:
+#
+# AX_COMPARE_VERSION([3.15.7],[lt],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[lt],[3.15.8])
+#
+# would both be true.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq],[3.15.8])
+# AX_COMPARE_VERSION([3.15],[gt],[3.15.8])
+#
+# would both be false.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq2],[3.15.8])
+#
+# would be true because it is only comparing two minor versions.
+#
+# AX_COMPARE_VERSION([3.15.7],[eq0],[3.15])
+#
+# would be true because it is only comparing the lesser number of minor
+# versions of the two values.
+#
+# Note: The characters that separate the version numbers do not matter. An
+# empty string is the same as version 0. OP is evaluated by autoconf, not
+# configure, so must be a string, not a variable.
+#
+# The author would like to acknowledge Guido Draheim whose advice about
+# the m4_case and m4_ifvaln functions make this macro only include the
+# portions necessary to perform the specific comparison specified by the
+# OP argument in the final configure script.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Tim Toolan <toolan@ele.uri.edu>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+dnl #########################################################################
+AC_DEFUN([AX_COMPARE_VERSION], [
+ AC_REQUIRE([AC_PROG_AWK])
+
+ # Used to indicate true or false condition
+ ax_compare_version=false
+
+ # Convert the two version strings to be compared into a format that
+ # allows a simple string comparison. The end result is that a version
+ # string of the form 1.12.5-r617 will be converted to the form
+ # 0001001200050617. In other words, each number is zero padded to four
+ # digits, and non digits are removed.
+ AS_VAR_PUSHDEF([A],[ax_compare_version_A])
+ A=`echo "$1" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ AS_VAR_PUSHDEF([B],[ax_compare_version_B])
+ B=`echo "$3" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
+ -e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
+ -e 's/[[^0-9]]//g'`
+
+ dnl # In the case of le, ge, lt, and gt, the strings are sorted as necessary
+ dnl # then the first line is used to determine if the condition is true.
+ dnl # The sed right after the echo is to remove any indented white space.
+ m4_case(m4_tolower($2),
+ [lt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [gt],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/false/;s/x${B}/true/;1q"`
+ ],
+ [le],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],
+ [ge],[
+ ax_compare_version=`echo "x$A
+x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/true/;s/x${B}/false/;1q"`
+ ],[
+ dnl Split the operator from the subversion count if present.
+ m4_bmatch(m4_substr($2,2),
+ [0],[
+ # A count of zero means use the length of the shorter version.
+ # Determine the number of characters in A and B.
+ ax_compare_version_len_A=`echo "$A" | $AWK '{print(length)}'`
+ ax_compare_version_len_B=`echo "$B" | $AWK '{print(length)}'`
+
+ # Set A to no more than B's length and B to no more than A's length.
+ A=`echo "$A" | sed "s/\(.\{$ax_compare_version_len_B\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(.\{$ax_compare_version_len_A\}\).*/\1/"`
+ ],
+ [[0-9]+],[
+ # A count greater than zero means use only that many subversions
+ A=`echo "$A" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ B=`echo "$B" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
+ ],
+ [.+],[
+ AC_WARNING(
+ [illegal OP numeric parameter: $2])
+ ],[])
+
+ # Pad zeros at end of numbers to make same length.
+ ax_compare_version_tmp_A="$A`echo $B | sed 's/./0/g'`"
+ B="$B`echo $A | sed 's/./0/g'`"
+ A="$ax_compare_version_tmp_A"
+
+ # Check for equality or inequality as necessary.
+ m4_case(m4_tolower(m4_substr($2,0,2)),
+ [eq],[
+ test "x$A" = "x$B" && ax_compare_version=true
+ ],
+ [ne],[
+ test "x$A" != "x$B" && ax_compare_version=true
+ ],[
+ AC_WARNING([illegal OP parameter: $2])
+ ])
+ ])
+
+ AS_VAR_POPDEF([A])dnl
+ AS_VAR_POPDEF([B])dnl
+
+ dnl # Execute ACTION-IF-TRUE / ACTION-IF-FALSE.
+ if test "$ax_compare_version" = "true" ; then
+ m4_ifvaln([$4],[$4],[:])dnl
+ m4_ifvaln([$5],[else $5])dnl
+ fi
+]) dnl AX_COMPARE_VERSION
diff --git a/autoconf-archive/ax_prog_python_version.m4 b/autoconf-archive/ax_prog_python_version.m4
new file mode 100644
index 0000000..628a3e4
--- /dev/null
+++ b/autoconf-archive/ax_prog_python_version.m4
@@ -0,0 +1,66 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_prog_python_version.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PROG_PYTHON_VERSION([VERSION],[ACTION-IF-TRUE],[ACTION-IF-FALSE])
+#
+# DESCRIPTION
+#
+# Makes sure that python supports the version indicated. If true the shell
+# commands in ACTION-IF-TRUE are executed. If not the shell commands in
+# ACTION-IF-FALSE are run. Note if $PYTHON is not set (for example by
+# running AC_CHECK_PROG or AC_PATH_PROG) the macro will fail.
+#
+# Example:
+#
+# AC_PATH_PROG([PYTHON],[python])
+# AX_PROG_PYTHON_VERSION([2.4.4],[ ... ],[ ... ])
+#
+# This will check to make sure that the python you have supports at least
+# version 2.4.4.
+#
+# NOTE: This macro uses the $PYTHON variable to perform the check.
+# AX_WITH_PYTHON can be used to set that variable prior to running this
+# macro. The $PYTHON_VERSION variable will be valorized with the detected
+# version.
+#
+# LICENSE
+#
+# Copyright (c) 2009 Francesco Salvestrini <salvestrini@users.sourceforge.net>
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 11
+
+AC_DEFUN([AX_PROG_PYTHON_VERSION],[
+ AC_REQUIRE([AC_PROG_SED])
+ AC_REQUIRE([AC_PROG_GREP])
+
+ AS_IF([test -n "$PYTHON"],[
+ ax_python_version="$1"
+
+ AC_MSG_CHECKING([for python version])
+ changequote(<<,>>)
+ python_version=`$PYTHON -V 2>&1 | $GREP "^Python " | $SED -e 's/^.* \([0-9]*\.[0-9]*\.[0-9]*\)/\1/'`
+ changequote([,])
+ AC_MSG_RESULT($python_version)
+
+ AC_SUBST([PYTHON_VERSION],[$python_version])
+
+ AX_COMPARE_VERSION([$ax_python_version],[le],[$python_version],[
+ :
+ $2
+ ],[
+ :
+ $3
+ ])
+ ],[
+ AC_MSG_WARN([could not find the python interpreter])
+ $3
+ ])
+])
diff --git a/autoconf-archive/ax_python_module.m4 b/autoconf-archive/ax_python_module.m4
new file mode 100644
index 0000000..f182c48
--- /dev/null
+++ b/autoconf-archive/ax_python_module.m4
@@ -0,0 +1,56 @@
+# ===========================================================================
+# http://www.gnu.org/software/autoconf-archive/ax_python_module.html
+# ===========================================================================
+#
+# SYNOPSIS
+#
+# AX_PYTHON_MODULE(modname[, fatal, python])
+#
+# DESCRIPTION
+#
+# Checks for Python module.
+#
+# If fatal is non-empty then absence of a module will trigger an error.
+# The third parameter can either be "python" for Python 2 or "python3" for
+# Python 3; defaults to Python 3.
+#
+# LICENSE
+#
+# Copyright (c) 2008 Andrew Collier
+#
+# Copying and distribution of this file, with or without modification, are
+# permitted in any medium without royalty provided the copyright notice
+# and this notice are preserved. This file is offered as-is, without any
+# warranty.
+
+#serial 8
+
+AU_ALIAS([AC_PYTHON_MODULE], [AX_PYTHON_MODULE])
+AC_DEFUN([AX_PYTHON_MODULE],[
+ if test -z $PYTHON;
+ then
+ if test -z "$3";
+ then
+ PYTHON="python3"
+ else
+ PYTHON="$3"
+ fi
+ fi
+ PYTHON_NAME=`basename $PYTHON`
+ AC_MSG_CHECKING($PYTHON_NAME module: $1)
+ $PYTHON -c "import $1" 2>/dev/null
+ if test $? -eq 0;
+ then
+ AC_MSG_RESULT(yes)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=yes
+ else
+ AC_MSG_RESULT(no)
+ eval AS_TR_CPP(HAVE_PYMOD_$1)=no
+ #
+ if test -n "$2"
+ then
+ AC_MSG_ERROR(failed to find required module $1)
+ exit 1
+ fi
+ fi
+])
diff --git a/configure.ac b/configure.ac
index fa87105..9d61051 100644
--- a/configure.ac
+++ b/configure.ac
@@ -14,6 +14,18 @@ AC_CONFIG_HEADER([config.h])
AC_CONFIG_SRCDIR([README])
AC_CONFIG_MACRO_DIR([m4])
+dnl Include some autoconf macros to check for python modules.
+dnl
+dnl These macros are coming from the autoconf archive at
+dnl http://www.gnu.org/software/autoconf-archive
+
+dnl This one is for the AX_PYTHON_MODULE() macro.
+m4_include([autoconf-archive/ax_python_module.m4])
+
+dnl These two below are for the AX_PROG_PYTHON_VERSION() module.
+m4_include([autoconf-archive/ax_compare_version.m4])
+m4_include([autoconf-archive/ax_prog_python_version.m4])
+
AM_INIT_AUTOMAKE([1.11.1 foreign subdir-objects tar-ustar parallel-tests])
AM_MAINTAINER_MODE([enable])
@@ -76,6 +88,12 @@ AC_ARG_ENABLE([bash-completion],
ENABLE_BASH_COMPLETION=$enableval,
ENABLE_BASH_COMPLETION=auto)
+AC_ARG_ENABLE([fedabipkgdiff],
+ AS_HELP_STRING([--enable-fedabipkgdiff=yes|no|auto],
+ [enable the fedabipkgdiff tool]),
+ ENABLE_FEDABIPKGDIFF=$enableval,
+ ENABLE_FEDABIPKGDIFF=auto)
+
dnl *************************************************
dnl check for dependencies
dnl *************************************************
@@ -219,6 +237,68 @@ fi
AM_CONDITIONAL(ENABLE_BASH_COMPLETION, test x$ENABLE_BASH_COMPLETION = xyes)
+dnl if --enable-fedabipkgdiff has the 'auto' value, then check for the required
+dnl python modules. If they are present, then enable the fedabipkgdiff program.
+dnl If they are not then disable the program.
+dnl
+dnl If --enable-fedabipkgdiff has the 'yes' value, then check for the required
+dnl python modules and whatever dependency fedabipkgdiff needs. If they are
+dnl not present then the configure script will error out.
+
+if test x$ENABLE_FEDABIPKGDIFF = xauto -o x$ENABLE_FEDABIPKGDIFF = xyes; then
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=yes
+else
+ CHECK_DEPS_FOR_FEDABIPKGDIFF=no
+fi
+
+if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then
+ if test x$ENABLE_FEDABIPKGDIFF = xyes; then
+ FATAL=yes
+ fi
+
+ AC_PATH_PROG(WGET, wget, no)
+
+ if test x$WGET = x$no; then
+ AC_MSG_ERROR(could not find the wget program)
+ fi
+
+ # The minimal python version we want to support is 2.6.6 because EL6
+ # distributions have that version installed.
+ MINIMAL_PYTHON_VERSION="2.6.6"
+
+ AC_PATH_PROG(PYTHON, python, no)
+ AX_PROG_PYTHON_VERSION($MINIMAL_PYTHON_VERSION,
+ [MINIMAL_PYTHON_VERSION_FOUND=yes],
+ [MINIMAL_PYTHON_VERSION_FOUND=no])
+
+ if test x$MINIMAL_PYTHON_VERSION_FOUND = xno; then
+ AC_MSG_ERROR([could not find a python program of version at least $MINIMAL_PYTHON_VERSION])
+ fi
+
+ AX_PYTHON_MODULE(argparse, $FATAL, python2)
+ AX_PYTHON_MODULE(glob, $FATAL, python2)
+ AX_PYTHON_MODULE(logging, $FATAL, python2)
+ AX_PYTHON_MODULE(os, $FATAL, python2)
+ AX_PYTHON_MODULE(re, $FATAL, python2)
+ AX_PYTHON_MODULE(shlex, $FATAL, python2)
+ AX_PYTHON_MODULE(subprocess, $FATAL, python2)
+ AX_PYTHON_MODULE(sys, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(urlparse, $FATAL, python2)
+ AX_PYTHON_MODULE(itertools, $FATAL, python2)
+ AX_PYTHON_MODULE(shutil, $FATAL, python2)
+ AX_PYTHON_MODULE(unittest, $FATAL, python2)
+ AX_PYTHON_MODULE(koji, $FATAL, python2)
+ AX_PYTHON_MODULE(mock, $FATAL, python2)
+ ENABLE_FEDABIPKGDIFF=yes
+
+ if test x$ENABLE_FEDABIPKGDIFF != xyes; then
+ ENABLE_FEDABIPKGDIFF=no
+ fi
+fi
+
+AM_CONDITIONAL(ENABLE_FEDABIPKGDIFF, test x$ENABLE_FEDABIPKGDIFF = xyes)
+
dnl Check for dependency: libzip
LIBZIP_VERSION=0.10.1
@@ -361,7 +441,10 @@ libabigail.pc
bash-completion/Makefile])
dnl Some test scripts are generated by autofoo.
-AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh], [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestcanonicalizetypes.sh],
+ [chmod +x tests/runtestcanonicalizetypes.sh])
+AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py],
+ [chmod +x tests/runtestfedabipkgdiff.py])
AC_OUTPUT
@@ -384,6 +467,7 @@ AC_MSG_NOTICE([
Enable deb support in abipkgdiff : ${ENABLE_DEB}
Enable GNU tar archive support in abipkgdiff : ${ENABLE_TAR}
Enable bash completion : ${ENABLE_BASH_COMPLETION}
+ Enable fedabipkgdiff : ${ENABLE_FEDABIPKGDIFF}
Generate html apidoc : ${ENABLE_APIDOC}
Generate html manual : ${ENABLE_MANUAL}
])
diff --git a/doc/manuals/Makefile.am b/doc/manuals/Makefile.am
index f23d1ab..5128bc0 100644
--- a/doc/manuals/Makefile.am
+++ b/doc/manuals/Makefile.am
@@ -10,7 +10,8 @@ conf.py \
index.rst \
libabigail-concepts.rst \
libabigail-overview.rst \
-libabigail-tools.rst
+libabigail-tools.rst \
+fedabipkgdiff
# You can set these variables from the command line.
SPHINXOPTS =
diff --git a/doc/manuals/conf.py b/doc/manuals/conf.py
index 2a7019f..a0967a2 100644
--- a/doc/manuals/conf.py
+++ b/doc/manuals/conf.py
@@ -219,6 +219,7 @@ man_pages = [
('abidw', 'abidw', u'serialize the ABI of an ELF file', [u'Dodji Seketeli'], 1),
('abilint', 'abilint', u'validate an abigail ABI representation', [u'Dodji Seketeli'], 1),
('abicompat', 'abicompat', u'check ABI compatibility', [u'Dodji Seketeli'], 1),
+ ('fedabipkgdiff', 'fedabipkgdiff', u'compare ABIs of Fedora packages', [u'Chenxiong Qi'], 1),
]
# If true, show URL addresses after external links.
diff --git a/doc/manuals/fedabipkgdiff.rst b/doc/manuals/fedabipkgdiff.rst
new file mode 100644
index 0000000..69e2368
--- /dev/null
+++ b/doc/manuals/fedabipkgdiff.rst
@@ -0,0 +1,108 @@
+.. _fedabipkgdiff_label:
+
+==============
+fedabipkgdiff
+==============
+
+``fedabipkgdiff`` compares ABIs of Fedora packages. It's a convenient way for
+Fedora packagers to run ``abipkgdiff`` against two RPM packages with their debug
+information packages. With well specified options, ``fedabipkgdiff`` is able to
+help packagers to find corresponding RPM packages from Koji, and then, if found,
+download them, and run ``abipkgdiff`` eventually to report possible ABI changes.
+
+
+.. _fedabipkgdiff_invocation_label:
+
+Invocation
+==========
+
+::
+
+ fedabipkgdiff [option] <NVR> ...
+
+
+.. _fedabipkgdiff_options_label:
+
+Options
+=======
+
+ * ``--help | -h``
+
+ Display a short help about the command and exit.
+
+ * ``--dry-run``
+
+ Don't actually run abipkgdiff. The commands that should be run will be sent
+ to stdout.
+
+ * ``--debug``
+
+ Run in debug mode, to show very detail debug output of each method
+ invocation and input paramters and returned result.
+
+ * ``--traceback``
+
+ Show traceback when there is an exception thrown. This could be useful for
+ developers to know what and where the error is.
+
+ * ``--server`` <URL>
+
+ URL of koji XMLRPC service. Default is http://koji.fedoraproject.org/kojihub
+
+ * ``--topdir`` <URL>
+
+ URL for RPM files access. Default is https://kojipkgs.fedoraproject.org
+
+ * ``--from`` <distro>
+
+ Baseline Fedora distrobution from where to find proper build that is used
+ for comparison. distro could be any valid value of RPM macro ``%{?dist}``
+ for Fedora, for example, ``fc4``, ``fc23``, ``fc25``.
+
+ * ``--to`` <distro>
+
+ The Fedora distrobution from where to find a proper build that is compared
+ with the baseline specified by option ``--from``.
+
+
+.. _fedabipkgdiff_return_value_label:
+
+Return value
+============
+
+The exit code of the ``abipkgdiff`` depends on the :ref:`return code
+<abipkgdiff_return_value_label>` from underlying abipkgdiff. The exit code is
+either 0 if all ``abipkgdiff`` invocations succeed and no ABI changes between
+packages, or non-zero that is returned by the last ``abipkgdiff`` invocation.
+
+
+.. _fedabipkgdiff_usage_example_label:
+
+
+Use cases
+=========
+
+Several cases ``fedabipkgdiff`` is supporting currectly are shown here.
+
+ 1. Compare ABIs of local package with latest stable package from Koji.
+ Suppose, you have built packages for httpd, and you would like to compare
+ the ABI of this locally built package with the latest stable package
+ built for Fedora 23. ::
+
+ $ fedabipkgdiff --from fc23 ./httpd-2.4.18-2.fc24.x86_64.rpm
+
+ 2. Compare ABIs of package httpd between Fedora 23 and 24. ::
+
+ $ fedabipkgdiff --from fc23 --to fc24 httpd
+
+ 3. Compare ABIs of package httpd designated by name, version and release,
+ even with a specific arch, for example x86_64. ::
+
+ $ fedabipkgdiff httpd-2.8.14.fc23 httpd-2.8.14.fc24
+ $ fedabipkgdiff httpd-2.8.14.fc23.x86_64 httpd-2.8.14.fc24.x86_64
+
+ 4. Within case 3, in addition, packager is also able to compare all packages
+ rather than specified package only. Obviously, both noarch and src packages
+ are excluded. ::
+
+ $ fedabipkgdiff --all-subpackages httpd-2.8.14.fc23 httpd-2.8.14.fc24
diff --git a/doc/manuals/libabigail-tools.rst b/doc/manuals/libabigail-tools.rst
index 2f7f4c1..d3d2492 100644
--- a/doc/manuals/libabigail-tools.rst
+++ b/doc/manuals/libabigail-tools.rst
@@ -20,3 +20,4 @@ Tools manuals
abicompat
abidw
abilint
+ fedabipkgdiff
diff --git a/tests/Makefile.am b/tests/Makefile.am
index caf49e6..953dfef 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -31,9 +31,10 @@ runtestlookupsyms \
runtestaltdwarf \
runtestcorediff \
runtestabidiffexit \
+runtestfedabipkgdiff.py \
$(CXX11_TESTS)
-EXTRA_DIST = runtestcanonicalizetypes.sh.in
+EXTRA_DIST = runtestcanonicalizetypes.sh.in runtestfedabipkgdiff.py.in
CLEANFILES = \
runtestcanonicalizetypes.output.txt \
runtestcanonicalizetypes.output.final.txt
@@ -114,6 +115,9 @@ printdifftree_LDADD = $(top_builddir)/src/libabigail.la
runtestcanonicalizetypes_sh_SOURCES =
runtestcanonicalizetypes.sh$(EXEEXT):
+runtestfedabipkgdiff_py_SOURCES =
+runtestfedabipkgdiff.py$(EXEEXT):
+
AM_CPPFLAGS=-I${abs_top_srcdir}/include \
-I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC
diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in
new file mode 100755
index 0000000..84a4320
--- /dev/null
+++ b/tests/runtestfedabipkgdiff.py.in
@@ -0,0 +1,634 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library (libabigail). This library is
+# free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 3, or (at your option) any
+# later version.
+#
+# This library is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public
+# License along with this program; see the file COPYING-GPLV3. If
+# not, see <http:#www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import os
+import itertools
+import unittest
+import tempfile
+import shutil
+
+import koji
+
+"""
+This test harness tests various global methods and classes within
+tools/fedabipkgdiff.
+"""
+
+try:
+ from mock import patch
+except ImportError:
+ import sys
+ print >>sys.stderr, \
+ 'mock is required to run tests. Please install before running tests.'
+ sys.exit(1)
+
+import imp
+# Import the fedabipkgdiff program file from the source directory.
+fedabipkgdiff_mod = imp.load_source('fedabidiff',
+ '@top_srcdir@/tools/fedabipkgdiff')
+
+# Used to generate integer values (greater or equal to zero) in
+# RunAbipkgdiffTest.test_partial_failure, those values simulate return code
+# from run_abipkgdiff. To represent partial failure, counter must start from 0.
+counter = itertools.count(0)
+
+# prefix for creating a temporary file or directory. The name would be
+# fedabipkgdiff-test-slkw3ksox
+temp_file_or_dir_prefix = 'fedabipkgdiff-test-'
+
+
+class UtilsTest(unittest.TestCase):
+
+ def test_is_distro_valid(self):
+ """Test is_fedora_distro method
+
+ is_fedora_distro aims to test if a string is a valid Fedora distro. I
+ don't see there is a general rule or format definition for such a
+ Fedora distro. I refer to second part of %{dist} splited by dot as the
+ reference. Generally, fc4, fc19, fc23 are valid ones, and el6, el7 are
+ also valid one currently.
+ """
+ distro = 'fc5'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'f5'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc23'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'fc234'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7'
+ self.assertTrue(fedabipkgdiff_mod.is_distro_valid(distro))
+
+ distro = 'el7_2'
+ self.assertFalse(fedabipkgdiff_mod.is_distro_valid(distro))
+
+
+class RPMTest(unittest.TestCase):
+ """Test case for RPM class
+
+ RPM class is a class wrapping a underlying dict object represeting a RPM
+ information, that is returned from Koji XMLRPC APIs.
+
+ This test aims to test the class to see if RPM attributes is accessible in
+ Python class attribute way, and if a RPM is specific type of RPM, for
+ example, if it's a debuginfo.
+ """
+
+ def setUp(self):
+ """Setup test data for testing RPM class
+
+ According to the tests, it's unnecessary to contruct a complete dict
+ containing full RPM information. So, only part of of them is
+ enough. This test case only need name, version, release, and arch.
+
+ In case Koji changes the name of name, version, release or arch in the
+ future to express same meaning individually, (I don't think it could
+ happen), please update there also.
+ """
+
+ # Argument passed to RPM.__init__ to construct a RPM class object, that
+ # represents a debuginfo RPM.
+ self.debuginfo_rpm_info = {
+ 'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+
+ # Argument passed to RPM.__init__ to construct a RPM class object, that
+ # represents a RPM.
+ self.rpm_info = {
+ 'arch': 'x86_64',
+ 'name': 'httpd',
+ 'release': '1.fc22',
+ 'version': '2.4.18'
+ }
+
+ def test_attribute_access(self):
+ """Ensure wrapped RPM information is accessible via attribute"""
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertEquals(self.debuginfo_rpm_info['arch'], rpm.arch)
+ self.assertEquals(self.debuginfo_rpm_info['name'], rpm.name)
+ self.assertEquals(self.debuginfo_rpm_info['release'], rpm.release)
+ self.assertEquals(self.debuginfo_rpm_info['version'], rpm.version)
+
+ def test_raise_error_if_name_not_exist(self):
+ """
+ Ensure AttributeError should be raised when accessing a non-existent
+ attribute
+ """
+ rpm = fedabipkgdiff_mod.RPM({})
+ try:
+ rpm.xxxxx
+ except AttributeError:
+ # Succeed, exit normally
+ return
+ self.fail('AttributeError should be raised, but not.')
+
+ def test_is_debuginfo(self):
+ """Ensure to return True if a RPM's name contains -debuginfo"""
+ rpm = fedabipkgdiff_mod.RPM(self.debuginfo_rpm_info)
+ self.assertTrue(rpm.is_debuginfo)
+
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertFalse(rpm.is_debuginfo)
+
+ def test_nvra(self):
+ """
+ Ensure value from RPM.nvra is parsable and contains correct value from
+ underlying RPM information
+ """
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ nvra = koji.parse_NVRA(rpm.nvra)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ def test_str_representation(self):
+ """
+ Enforce a RPM object has same string represetation as underlying
+ wrapped rpm information that is a dict object.
+ """
+ rpm = fedabipkgdiff_mod.RPM(self.rpm_info)
+ self.assertEquals(str(self.rpm_info), str(rpm))
+
+
+class LocalRPMTest(unittest.TestCase):
+ """Test case for LocalRPM class
+
+ Because LocalRPM inherits from RPM, all tests against RPM class are also
+ applied to LocalRPM, so I don't repeat them again here. This test case
+ mainly focus on the abilities against files on the local disk.
+ """
+
+ def setUp(self):
+ # A RPM filename that simulates a RPM file that is stored somewhere on
+ # the disk.
+ # This is the only argument passed to LocalRPM.__init__ to initialize
+ # an object.
+ self.filename = 'httpd-2.4.18-1.fc22.x86_64.rpm'
+
+ def test_file_parser_without_path(self):
+ """Ensure LocalRPM can get RPM information from a filename
+
+ LocalRPM gets name, version, release, and arch of a RPM by parsing the
+ passed filename to __init__ method. Then, all these information is
+ accessible via LocalRPM name, version, release, and arch attribute.
+
+ A filename either with an absolute path, relative path, or without a
+ path, LocalRPM should be able to find these files and get correct
+ information by removing the potential present path. For example, by
+ giving following filenames,
+
+ - httpd-2.4.18-1.fc22.x86_64.rpm
+ - artifacts/httpd-2.4.18-1.fc22.x86_64.rpm
+ - /mnt/koji/packages/httpd/2.4.18/1.fc22/httpd-2.4.18-1.fc22.x86_64.rpm
+
+ LocalRPM has to determine the necessary RPM information from
+ httpd-2.4.18-1.fc22.x86_64.rpm
+
+ Without specifying path in the filename, it usually means LocalRPM
+ should find the RPM file relative to current working directory. So, no
+ need of additional test against a filename with a relative path.
+ """
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+
+ full_filename = os.path.join('/', 'tmp', self.filename)
+ rpm = fedabipkgdiff_mod.LocalRPM(full_filename)
+ nvra = koji.parse_NVRA(self.filename)
+ self.assertEquals(nvra['name'], rpm.name)
+ self.assertEquals(nvra['version'], rpm.version)
+ self.assertEquals(nvra['release'], rpm.release)
+ self.assertEquals(nvra['arch'], rpm.arch)
+ self.assertEquals(full_filename, rpm.downloaded_file)
+
+ @patch('os.path.exists')
+ def test_find_existent_debuginfo(self, mock_exists):
+ """Ensure LocalRPM can find an associated existent debuginfo RPM
+
+ Currently, find_debuginfo is only able to find associated debuginfo RPM
+ from the directory where local RPM resides. This test works for this
+ case at this moment. If there is a requirement to allow find debuginfo
+ RPM from somewhere else, any level of subdirectory for instance, add
+ new test case for that, and update these words you are reading :)
+ """
+ mock_exists.return_value = True
+
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertTrue(isinstance(rpm, fedabipkgdiff_mod.LocalRPM))
+
+ nvra = koji.parse_NVRA(self.filename)
+ expected_debuginfo = fedabipkgdiff_mod.LocalRPM(
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % nvra)
+ debuginfo = rpm.find_debuginfo()
+ self.assertEquals(expected_debuginfo.name, debuginfo.name)
+ self.assertEquals(expected_debuginfo.version, debuginfo.version)
+ self.assertEquals(expected_debuginfo.release, debuginfo.release)
+
+ def test_find_non_existent_debuginfo(self):
+ """Ensure to return None if cannot find associated debuginfo RPM
+
+ os.path.exists is not mocked, that is because the associated debuginfo
+ RPM of httpd-2.4.18-1.fc22.x86_64.rpm given in setUp must be
+ non-existed during this test's run.
+ """
+ rpm = fedabipkgdiff_mod.LocalRPM(self.filename)
+ self.assertEquals(None, rpm.find_debuginfo())
+
+
+class RunAbipkgdiffTest(unittest.TestCase):
+ """Test case for method run_abipkgdiff
+
+ Method run_abipkgdiff accepts package informations and passes them to and
+ run abipkgdiff command line utility. Since run_abipkgdiff does not catch
+ output to either standard output or standard error, and only returns the
+ return code that is returned from underlying abipkgdiff, these various test
+ cases test whether run_abipkgdiff is able to return the return code
+ correctly.
+ """
+
+ def setUp(self):
+ """Define packages information for calling run_abipkgdiff method
+
+ Due to the tests just care about the return code from underlying
+ abipkgdiff, only partial attributes of a RPM is required. That means,
+ it's unnecessary to give a full dict representing a complete RPM, just
+ build_id, name, version, release, and arch.
+
+ Full RPM information is not required. For this test case, only partial
+ information arch, build_id, name, release, and version are enough.
+ """
+
+ # Used for testing the case of running abipkgdiff against one RPM
+ self.pkg1_single_info = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ })
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_single_info = self.pkg1_single_info.copy()
+
+ # Used for testing the case of running abipkgdiff against multiple RPMs
+ self.pkg1_infos = {
+ 'i686': [
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'i686',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'x86_64': [
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'x86_64',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ 'armv7hl': [
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ fedabipkgdiff_mod.RPM({'arch': 'armv7hl',
+ 'build_id': 720222,
+ 'name': 'httpd-debuginfo',
+ 'release': '2.fc24',
+ 'version': '2.4.18',
+ }),
+ ],
+ }
+
+ # Whatever the concrete content of pkg2_infos is, so just make a copy
+ # from self.pkg1_infos
+ self.pkg2_infos = self.pkg1_infos.copy()
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_success(self, mock_abipkgdiff):
+ """
+ Ensure run_abipkgdiff returns 0 when it succeeds to run against one or
+ more packages.
+ """
+ mock_abipkgdiff.return_value = 0
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(0, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(0, result)
+
+ @patch('fedabidiff.abipkgdiff')
+ def test_all_failure(self, mock_abipkgdiff):
+ """
+ Ensure run_abipkgdiff returns the return code from underlying
+ abipkgdiff when all calls to abipkgdiff fails against one or more
+ packages.
+ """
+ mock_abipkgdiff.return_value = 4
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_single_info,
+ self.pkg2_single_info)
+ self.assertEquals(4, result)
+
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertEquals(4, result)
+
+ @patch('fedabidiff.abipkgdiff', new=lambda param1, param2: counter.next())
+ def test_partial_failure(self):
+ """
+ Ensure run_abipkgdiff returns non-zero when partial calls to
+ run_abipkgdiff succeed
+
+ abipkgdiff is mocked in order to simulte the partial success
+ calls. Why? That is because, counter starts from 0. So, it will
+ generate 0, 1, 2, ...
+ """
+ result = fedabipkgdiff_mod.run_abipkgdiff(self.pkg1_infos,
+ self.pkg2_infos)
+ self.assertTrue(result > 0)
+
+
+class DownloadRPMTest(unittest.TestCase):
+ """Test case for download_rpm
+
+ Download a remote file, which is a local file simulating a remote file with
+ scheme file://, for example file:///tmp/a.txt, to download directory.
+ """
+
+ def setUp(self):
+ # Create a remote file for testing download of this file
+ self.fd, self.remote_filename = tempfile.mkstemp(
+ prefix=temp_file_or_dir_prefix)
+ # Whatever the content is, this case does not care about. Close it
+ # immediately.
+ os.close(self.fd)
+
+ # Used as a fake download directory to mock get_download_dir method
+ self.download_dir = tempfile.mkdtemp(prefix=temp_file_or_dir_prefix)
+
+ def tearDown(self):
+ os.remove(self.remote_filename)
+ shutil.rmtree(self.download_dir)
+
+ def make_remote_file_url(self):
+ """Make URL of remote file that is used for downloading this file"""
+ return 'file://{}'.format(self.remote_filename)
+
+ def make_nonexistent_remote_file_url(self):
+ """Return URL to a non-existent remote file"""
+ return os.path.join(self.make_remote_file_url(), 'nonexistent-file')
+
+ @patch('__main__.fedabipkgdiff_mod.get_download_dir')
+ def test_succeed_to_download_a_rpm(self, mock_get_download_dir):
+ """Enusre True is returned if curl succeeds to download remote file
+
+ Download remote file to a fake download directory. Ensure everything is
+ okay, and return value from download_rpm should be truth.
+ """
+ mock_get_download_dir.return_value = self.download_dir
+
+ url = self.make_remote_file_url()
+ ret = fedabipkgdiff_mod.download_rpm(url)
+ self.assertTrue(ret)
+
+ @patch('__main__.fedabipkgdiff_mod.get_download_dir')
+ def test_failed_to_download_a_rpm(self, mock_get_download_dir):
+ """Ensure False is returned if curl fails to download remote file
+
+ Download remote file to a fake download directory. But, making
+ something wrong to cause download_rpm returns false.
+ """
+ mock_get_download_dir.return_value = self.download_dir
+
+ url = self.make_nonexistent_remote_file_url()
+ ret = fedabipkgdiff_mod.download_rpm(url)
+ self.assertFalse(ret)
+
+
+class MockGlobalConfig(object):
+ """Used to mock global_config
+
+ Since tests do not parse options from command line, so this class is
+ helpful for tests to contain all potential parsed (simulated)
+ options.
+
+ Currently, only koji_server is required for running tests. If any new test
+ cases need others, please add them add as class attribute directly.
+ """
+ koji_server = fedabipkgdiff_mod.DEFAULT_KOJI_SERVER
+
+
+def mock_get_session():
+ """Used to mock get_session method to get mocked KojiSession instance"""
+ return MockKojiClientSession(baseurl=fedabipkgdiff_mod.DEFAULT_KOJI_SERVER)
+
+
+class MockKojiClientSession(object):
+ """Mock koji.ClientSession
+
+ This mock ClientSession aims to avoid touching a real Koji instance to
+ interact with XMLRPC APIs required by fedabipkgdiff.
+
+ For the tests within this module, methods do not necessarily to return
+ complete RPM and build information. So, if you need more additional
+ information, here is the right place to add them.
+ """
+
+ def __init__(self, *args, **kwargs):
+ """Accept arbitrary parameters but do nothing for this mock
+
+ Add this method, although it's not used by this mock ClientSession.
+ That is because, when initiate an instance of koji.ClientSession,
+ paramters, at least URL to kojihub, is required and passed.
+ """
+ self.args = args
+ self.kwargs = kwargs
+
+ def getPackage(self, *args, **kwargs):
+ return {
+ 'id': 1,
+ 'name': 'whatever a name of a package',
+ }
+
+ def listRPMs(self, *args, **kwargs):
+ return [{'arch': 'i686',
+ 'name': 'httpd-debuginfo',
+ 'nvr': 'httpd-debuginfo-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_session',
+ 'nvr': 'mod_session-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_proxy_html',
+ 'nvr': 'mod_proxy_html-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ldap',
+ 'nvr': 'mod_ldap-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'arch': 'i686',
+ 'name': 'mod_ssl',
+ 'nvr': 'mod_ssl-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'}]
+
+ def listBuilds(self, *args, **kwargs):
+ return [
+ {'build_id': 720222,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-2.fc24',
+ 'release': '2.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 708769,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc22',
+ 'release': '1.fc22',
+ 'version': '2.4.18'},
+ {'build_id': 708711,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc23',
+ 'release': '1.fc23',
+ 'version': '2.4.18'},
+ {'build_id': 705335,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.18-1.fc24',
+ 'release': '1.fc24',
+ 'version': '2.4.18'},
+ {'build_id': 704434,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc24',
+ 'release': '4.fc24',
+ 'version': '2.4.17'},
+ {'build_id': 704433,
+ 'name': 'httpd',
+ 'nvr': 'httpd-2.4.17-4.fc23',
+ 'release': '4.fc23',
+ 'version': '2.4.17'},
+ ]
+
+
+class GetPackageLatestBuildTest(unittest.TestCase):
+ """Test case for get_package_latest_build"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_get_latest_one(self):
+ """Ensure to get latest build of a package
+
+ Brew.listBuilds is mocked to return a series of builds that contains
+ a latest build, which is httpd-2.4.18-1.fc23, of package httpd for
+ Fedora 23, and that must be found and returned.
+ """
+ session = fedabipkgdiff_mod.get_session()
+ build = session.get_package_latest_build('httpd', 'fc23')
+ self.assertEquals('httpd-2.4.18-1.fc23', build['nvr'])
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('koji.ClientSession', new=MockKojiClientSession)
+ def test_cannot_find_a_latest_build_with_invalid_distro(self):
+ """
+ Ensure NoCompleteBuilds is raised when trying to find a latest build of
+ a package for unknown Fedora distribution.
+ """
+ session = fedabipkgdiff_mod.get_session()
+ self.assertRaises(fedabipkgdiff_mod.NoCompleteBuilds,
+ session.get_package_latest_build, 'httpd', 'xxxx')
+
+
+class BrewListRPMsTest(unittest.TestCase):
+ """Test case for Brew.listRPMs"""
+
+ @patch('fedabidiff.global_config', new=MockGlobalConfig)
+ @patch('fedabidiff.koji.ClientSession', new=MockKojiClientSession)
+ def test_select_specific_rpms(self):
+ """Ensure Brew.listRPMs can select RPMs by a specific selector
+
+ This test will select RPMs whose name starts with httpd, that is only
+ httpd and httpd-debuginfo RPMs are selected and returned.
+ """
+ session = fedabipkgdiff_mod.get_session()
+ selector = lambda rpm: rpm['name'].startswith('httpd')
+ rpms = session.listRPMs(buildID=1000, selector=selector)
+ self.assertTrue(
+ len(rpms) > 0,
+ 'More than one rpms should be selected. But, it\'s empty.')
+ for rpm in rpms:
+ self.assertTrue(rpm['name'] in ('httpd', 'httpd-debuginfo'),
+ '{0} should not be selected'.format(rpm['name']))
+
+
+if __name__ == '__main__':
+ unittest.main()
diff --git a/tools/Makefile.am b/tools/Makefile.am
index b855f41..0d96215 100644
--- a/tools/Makefile.am
+++ b/tools/Makefile.am
@@ -6,6 +6,12 @@ else
bin_PROGRAMS = abidiff abilint abidw abicompat abipkgdiff
endif
+if ENABLE_FEDABIPKGDIFF
+ bin_SCRIPTS = fedabipkgdiff
+else
+ noinst_SCRIPTS = fedabipkgdiff
+endif
+
noinst_PROGRAMS = abisym abinilint
if ENABLE_ZIP_ARCHIVE
diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff
new file mode 100755
index 0000000..e7c4785
--- /dev/null
+++ b/tools/fedabipkgdiff
@@ -0,0 +1,1050 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+# -*- Mode: Python
+#
+# Copyright (C) 2013-2016 Red Hat, Inc.
+#
+# This file is part of the GNU Application Binary Interface Generic
+# Analysis and Instrumentation Library (libabigail). This library is
+# free software; you can redistribute it and/or modify it under the
+# terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 3, or (at your option) any
+# later version.
+#
+# This library is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# General Public License for more details.
+#
+# You should have received a copy of the GNU General Public
+# License along with this program; see the file COPYING-GPLV3. If
+# not, see <http:#www.gnu.org/licenses/>.
+#
+# Author: Chenxiong Qi
+
+import argparse
+import logging
+import os
+import re
+import shlex
+import subprocess
+import sys
+
+from collections import namedtuple
+from itertools import groupby
+
+import xdg.BaseDirectory
+
+import koji
+
+# @file
+#
+# You might have known that abipkgdiff is a command line tool to compare two
+# RPM packages to find potential differences of ABI. This is really useful for
+# Fedora packagers and developers. Usually, excpet the RPM packages built
+# locally, if a packager wants to compare RPM packages he just built with
+# specific RPM packages that were already built and availabe in Koji,
+# fedabipkgdiff is the right tool for him.
+#
+# With fedabipkgdiff, packager is able to specify certain criteria to tell
+# fedabipkgdiff which RPM packages he wants to compare, then fedabipkgdiff will
+# find them, download them, and boom, run the abipkgdiff for you.
+#
+# Currently, fedabipkgdiff returns 0 if everything works well, otherwise, 1 if
+# something wrong.
+
+
+DEFAULT_KOJI_SERVER = 'http://koji.fedoraproject.org/kojihub'
+DEFAULT_KOJI_TOPDIR = 'https://kojipkgs.fedoraproject.org'
+
+# The working directory where to hold all data including downloaded RPM
+# packages Currently, it's not configurable and hardcode here. In the future
+# version of fedabipkgdiff, I'll make it configurable by users.
+HOME_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home,
+ os.path.splitext(os.path.basename(__file__))[0])
+
+# Used to construct abipkgdiff command line argument, package and associated
+# debuginfo package
+# fedabipkgdiff runs abipkgdiff in this form
+#
+# abipkgdiff \
+# --d1 /path/to/package1-debuginfo.rpm \
+# --d2 /path/to/package2-debuginfo.rpm \
+# /path/to/package1.rpm \
+# /path/to/package2.rpm
+#
+# PkgInfo is a two-elements tuple in format
+#
+# (/path/to/package1.rpm, /path/to/package1-debuginfo.rpm)
+#
+# So, before calling abipkgdiff, fedabipkgdiff must prepare and pass following
+# two package information
+#
+# (/path/to/package1.rpm, /path/to/package1-debuginfo.rpm)
+# (/path/to/package2.rpm, /path/to/package2-debuginfo.rpm)
+#
+PkgInfo = namedtuple('PkgInfo', 'package debuginfo_package')
+
+
+global_config = None
+pathinfo = None
+session = None
+
+# There is no way to configure the log format so far. I hope I would have time
+# to make it available so that if fedabipkgdiff is scheduled and run by some
+# service, the logs logged into log file is muc usable.
+logging.basicConfig(format='[%(levelname)s] %(message)s',
+ level=logging.CRITICAL)
+logger = logging.getLogger(os.path.basename(__file__))
+
+
+class KojiPackageNotFound(Exception):
+ """Package is not found in Koji"""
+
+
+class PackageNotFound(Exception):
+ """Package is not found locally"""
+
+
+class RpmNotFound(Exception):
+ """RPM is not found"""
+
+
+class NoBuildsError(Exception):
+ """No builds returned from a method to select specific builds"""
+
+
+class NoCompleteBuilds(Exception):
+ """No complete builds for a package
+
+ This is a serious problem, nothing can be done if there is no complete
+ builds for a package.
+ """
+
+
+class InvalidDistroError(Exception):
+ """Invalid distro error"""
+
+
+class CannotFindLatestBuildError(Exception):
+ """Cannot find latest build from a package"""
+
+
+def is_distro_valid(distro):
+ """Adjust if a distro is valid
+
+ Currently, check for Fedora and RHEL.
+
+ :param str distro: a string representing a distro value.
+ :return: True if distro is the one specific to Fedora, like fc24, el7.
+ "rtype: bool
+ """
+ return re.match(r'^(fc|el)\d{1,2}$', distro) is not None
+
+
+def log_call(func):
+ """A decorator that logs a method invocation
+
+ Method's name and all arguments, either positional or keyword arguments,
+ will be logged by logger.debug. Also, return value from the decorated
+ method will be logged just after the invocation is done.
+
+ This decorator does not catch any exception thrown from the decorated
+ method. If there is any exception thrown from decorated method, you can
+ catch them in the caller and obviously, no return value is logged.
+
+ :param callable func: a callable object to decorate
+ """
+ def proxy(*args, **kwargs):
+ logger.debug('Call %s, args: %s, kwargs: %s',
+ func.__name__,
+ args if args else '',
+ kwargs if kwargs else '')
+ result = func(*args, **kwargs)
+ logger.debug('Result from %s: %s', func.__name__, result)
+ return result
+ return proxy
+
+
+class RPM(object):
+ """Wrapper of RPM representing a RPM got from Koji
+
+ A RPM is returned from Koji XMLRPC API is in dict type. This wrapper makes
+ it eaiser to access all these properties in the way of object.property.
+ """
+
+ def __init__(self, rpm_info):
+ """Initialize a RPM object
+
+ :param dict rpm_info: a dict representing a RPM information got from
+ koji API, either listRPMs or getRPM
+ """
+ self.rpm_info = rpm_info
+
+ def __str__(self):
+ """Return the string representation of this RPM
+
+ Return the string representation of RPM information returned from Koji
+ directly so that RPM can be treated in same way.
+ """
+ return str(self.rpm_info)
+
+ def __getattr__(self, name):
+ """Access RPM information in the way of object.property
+
+ :param str name: the property name to access.
+ :raises AttributeError: if name is not one of keys of RPM information.
+ """
+ if name in self.rpm_info:
+ return self.rpm_info[name]
+ else:
+ raise AttributeError('No attribute name {0}'.format(name))
+
+ @property
+ def nvra(self):
+ """Return a RPM's N-V-R-A representation
+
+ An example: libabigail-1.0-0.8.rc4.1.fc23.x86_64
+ """
+ return '%(name)s-%(version)s-%(release)s.%(arch)s' % self.rpm_info
+
+ @property
+ def filename(self):
+ """Return a RPM file name
+
+ An example: libabigail-1.0-0.8.rc4.1.fc23.x86_64.rpm
+ """
+ return '{0}.rpm'.format(self.nvra)
+
+ @property
+ def is_debuginfo(self):
+ """Check if the name of the current RPM denotes a debug info package"""
+ return koji.is_debuginfo(self.rpm_info['name'])
+
+ @property
+ def download_url(self):
+ """Get the URL from where to download this RPM"""
+ build = session.getBuild(self.build_id)
+ return os.path.join(pathinfo.build(build), pathinfo.rpm(self.rpm_info))
+
+ @property
+ def downloaded_file(self):
+ """Get a pridictable downloaded file name with absolute path"""
+ # arch should be removed from the result returned from PathInfo.rpm
+ filename = os.path.basename(pathinfo.rpm(self.rpm_info))
+ return os.path.join(get_download_dir(), filename)
+
+ @property
+ def is_downloaded(self):
+ """Check if this RPM was already downloaded to local disk"""
+ return os.path.exists(self.downloaded_file)
+
+
+class LocalRPM(RPM):
+ """Representing a local RPM
+
+ Local RPM means the one that could be already downloaded or built from
+ where I can find it
+ """
+
+ def __init__(self, filename):
+ """Initialize local RPM with a filename
+
+ :param str filename: a filename pointing to a RPM file in local
+ disk. Note that, this file must not exist necessarily.
+ """
+ self.local_filename = filename
+ self.rpm_info = koji.parse_NVRA(os.path.basename(filename))
+
+ @property
+ def downloaded_file(self):
+ """Return filename of this RPM
+
+ Returned filename is just the one passed when initializing this RPM.
+
+ :return: filename of this RPM
+ :rtype: str
+ """
+ return self.local_filename
+
+ @property
+ def download_url(self):
+ raise NotImplementedError('LocalRPM has no URL to download')
+
+ @log_call
+ def find_debuginfo(self):
+ """Find debuginfo RPM package from a directory
+
+ :param str rpm_file: the rpm file name
+ :return: the absolute file name of the found debuginfo rpm
+ :rtype: str or None
+ """
+ search_dir = os.path.dirname(os.path.abspath(self.local_filename))
+ filename = \
+ '%(name)s-debuginfo-%(version)s-%(release)s.%(arch)s.rpm' % \
+ self.rpm_info
+ filename = os.path.join(search_dir, filename)
+ return LocalRPM(filename) if os.path.exists(filename) else None
+
+
+class Brew(object):
+ """Interface to Koji XMLRPC API with enhancements specific to fedabipkgdiff
+
+ kojihub XMLRPC APIs are well-documented in koji's source code. For more
+ details information, please refer to class RootExports within kojihub.py.
+
+ For details of APIs used within fedabipkgdiff, refer to from line
+
+ https://pagure.io/koji/blob/master/f/hub/kojihub.py#_7835
+ """
+
+ def __init__(self, baseurl):
+ """Initialize Brew
+
+ :param str baseurl: the kojihub URL to initialize a session, that is
+ used to access koji XMLRPC APIs.
+ """
+ self.session = koji.ClientSession(baseurl)
+
+ @log_call
+ def listRPMs(self, buildID=None, arches=None, selector=None):
+ """Get list of RPMs of a build from Koji
+
+ Call kojihub.listRPMs to get list of RPMs. Return selected RPMs without
+ changing each RPM information.
+
+ A RPM returned from listRPMs contains following keys:
+
+ - id
+ - name
+ - version
+ - release
+ - nvr (synthesized for sorting purposes)
+ - arch
+ - epoch
+ - payloadhash
+ - size
+ - buildtime
+ - build_id
+ - buildroot_id
+ - external_repo_id
+ - external_repo_name
+ - metadata_only
+ - extra
+
+ :param int buildID: id of a build from which to list RPMs.
+ :param arches: to restrict to list RPMs with specified arches.
+ :type arches: list or tuple
+ :param selector: called to determine if a RPM should be selected and
+ included in the final returned result. Selector must be a callable
+ object and accepts one parameter of a RPM.
+ :type selector: a callable object
+ :return: a list of RPMs, each of them is a dict object
+ :rtype: list
+ """
+ if selector:
+ assert hasattr(selector, '__call__'), 'selector must be callable.'
+ rpms = self.session.listRPMs(buildID=buildID, arches=arches)
+ if selector:
+ rpms = [rpm for rpm in rpms if selector(rpm)]
+ return rpms
+
+ @log_call
+ def getRPM(self, rpminfo):
+ """Get a RPM from koji
+
+ Call kojihub.getRPM, and returns the result directly without any
+ change.
+
+ When not found a RPM, koji.getRPM will return None, then
+ this method will raise RpmNotFound error immediately to claim what is
+ happening. I want to raise fedabipkgdiff specific error rather than
+ koji's GenericError and then raise RpmNotFound again, so I just simply
+ don't use strict parameter to call koji.getRPM.
+
+ :param rpminfo: rpminfo may be a N-V-R.A or a map containing name,
+ version, release, and arch. For example, file-5.25-5.fc24.x86_64, and
+ `{'name': 'file', 'version': '5.25', 'release': '5.fc24', 'arch':
+ 'x86_64'}`.
+ :type rpminfo: str or dict
+ :return: a map containing RPM information, that contains same keys as
+ method `Brew.listRPMs`.
+ :rtype: dict
+ :raises RpmNotFound: if a RPM cannot be found with rpminfo.
+ """
+ rpm = self.session.getRPM(rpminfo)
+ if rpm is None:
+ raise RpmNotFound('Cannot find RPM {0}'.format(args[0]))
+ return rpm
+
+ @log_call
+ def listBuilds(self, packageID, state=None, topone=None,
+ selector=None, order_by=None, reverse=None):
+ """Get list of builds from Koji
+
+ Call kojihub.listBuilds, and return selected builds without changing
+ each build information.
+
+ By default, only builds with COMPLETE state are queried and returns
+ afterwards.
+
+ :param int packageID: id of package to list builds from.
+ :param int state: build state. There are five states of a build in
+ Koji. fedabipkgdiff only cares about builds with COMPLETE state. If
+ state is omitted, builds with COMPLETE state are queried from Koji by
+ default.
+ :param bool topone: just return the top first build.
+ :param selector: a callable object used to select specific subset of
+ builds. Selector will be called immediately after Koji returns queried
+ builds. When each call to selector, a build is passed to
+ selector. Return True if select current build, False if not.
+ :type selector: a callable object
+ :param str order_by: the attribute name by which to order the builds,
+ for example, name, version, or nvr.
+ :param bool reverse: whether to order builds reversely.
+ :return: a list of builds, even if there is only one build.
+ :rtype: list
+ :raises TypeError: if selector is not callable, or if order_by is not a
+ string value.
+ """
+ if state is None:
+ state = koji.BUILD_STATES['COMPLETE']
+
+ if selector is not None and not hasattr(selector, '__call__'):
+ raise TypeError(
+ '{0} is not a callable object.'.format(str(selector)))
+
+ if order_by is not None and not isinstance(order_by, basestring):
+ raise TypeError('order_by {0} is invalid.'.format(order_by))
+
+ builds = self.session.listBuilds(packageId=packageID, state=state)
+ if selector is not None:
+ builds = [build for build in builds if selector(build)]
+ if order_by is not None:
+ # FIXME: is it possible to sort builds by using opts parameter of
+ # listBuilds
+ builds = sorted(builds,
+ key=lambda item: item[order_by],
+ reverse=reverse)
+ if topone:
+ builds = builds[0:1]
+
+ return builds
+
+ @log_call
+ def getPackage(self, name):
+ """Get a package from Koji
+
+ :param str name: a package name.
+ :return: a mapping containing package information. For example,
+ `{'id': 1, 'name': 'package'}`.
+ :rtype: dict
+ """
+ package = self.session.getPackage(name)
+ if package is None:
+ package = self.session.getPackage(name.rsplit('-', 1)[0])
+ if package is None:
+ raise KojiPackageNotFound(
+ 'Cannot find package {0}.'.format(name))
+ return package
+
+ @log_call
+ def getBuild(self, buildID):
+ """Get a build from Koji
+
+ Call kojihub.getBuild. Return got build directly without change.
+
+ :param int buildID: id of build to get from Koji.
+ :return: the found build. Return None, if not found a build with
+ buildID.
+ :rtype: dict
+ """
+ return self.session.getBuild(buildID)
+
+ @log_call
+ def get_rpm_build_id(self, name, version, release, arch=None):
+ """Get build ID that contains a RPM with specific nvra
+
+ If arch is not omitted, a RPM can be identified by its N-V-R-A.
+
+ If arch is omitted, name is used to get associated package, and then
+ to get the build.
+
+ Example:
+
+ >>> brew = Brew('url to kojihub')
+ >>> brew.get_rpm_build_id('httpd', '2.4.18', '2.fc24')
+ >>> brew.get_rpm_build_id('httpd', '2.4.18', '2.fc25', 'x86_64')
+
+ :param str name: name of a rpm
+ :param str version: version of a rpm
+ :param str release: release of a rpm
+ :param arch: arch of a rpm
+ :type arch: str or None
+ :return: id of the build from where the RPM is built
+ :rtype: dict
+ :raises KojiPackageNotFound: if name is not found from Koji if arch
+ is None.
+ """
+ if arch is None:
+ package = self.getPackage(name)
+ selector = lambda item: item['version'] == version and \
+ item['release'] == release
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector)
+ if not builds:
+ raise NoBuildsError(
+ 'No builds are selected from package {0}.'.format(
+ package['name']))
+ return builds[0]['build_id']
+ else:
+ rpm = self.getRPM({'name': name,
+ 'version': version,
+ 'release': release,
+ 'arch': arch,
+ })
+ return rpm['build_id']
+
+ @log_call
+ def get_package_latest_build(self, package_name, distro):
+ """Get latest build from a package
+
+ Example:
+
+ >>> brew = Brew('url to kojihub')
+ >>> brew.get_package_latest_build('httpd', 'fc24')
+
+ :param str package_name: from which package to get the latest build
+ :param str distro: which distro the latest build belongs to
+ :return: the found build
+ :rtype: dict or None
+ :raises NoCompleteBuilds: if there is no latest build of a package.
+ """
+ package = self.getPackage(package_name)
+ selector = lambda item: item['release'].find(distro) > -1
+
+ builds = self.listBuilds(packageID=package['id'],
+ selector=selector,
+ order_by='nvr',
+ reverse=True)
+ if not builds:
+ raise NoCompleteBuilds(
+ 'No complete builds of package {0}'.format(package_name))
+
+ return builds[0]
+
+ @log_call
+ def select_rpms_from_a_build(self, build_id, package_name, arches=None,
+ select_subpackages=None):
+ """Select specific RPMs within a build
+
+ RPMs could be filtered be specific criterias by the parameters.
+
+ By default, fedabipkgdiff requires RPM package and associated debuginfo
+ package, both of these two packages are selected, and noarch and src
+ are excluded.
+
+ :param int build_id: from which build to select rpms.
+ :param str package_name: which rpm to select that matches this name.
+ :param arches: which arches to select. If arches omits, rpms with all
+ arches except noarch and src will be selected.
+ :type arches: list, tuple or None
+ :param bool select_subpackages: indicate whether to select all RPMs
+ with specific arch from build.
+ :return: a list of RPMs returned from listRPMs
+ :rtype: list
+ """
+ excluded_arches = ('noarch', 'src')
+
+ def rpms_selector(package_name, excluded_arches):
+ return lambda rpm: \
+ rpm['arch'] not in excluded_arches and \
+ (rpm['name'] == package_name or
+ rpm['name'].endswith('-debuginfo'))
+
+ if select_subpackages:
+ selector = lambda rpm: rpm['arch'] not in excluded_arches
+ else:
+ selector = rpms_selector(package_name, excluded_arches)
+ rpm_infos = self.listRPMs(buildID=build_id,
+ arches=arches,
+ selector=selector)
+ return [RPM(rpm_info) for rpm_info in rpm_infos]
+
+ @log_call
+ def get_latest_built_rpms(self, package_name, distro, arches=None):
+ """Get RPMs from latest build of a package
+
+ :param str package_name: from which package to get the rpms
+ :param str distro: which distro the rpms belong to
+ :param arches: which arches the rpms belong to
+ :type arches: str or None
+ :return: the selected RPMs
+ :rtype: list
+ """
+ latest_build = self.get_package_latest_build(package_name, distro)
+ # Get rpm and debuginfo rpm from each arch
+ return self.select_rpms_from_a_build(latest_build['build_id'],
+ package_name,
+ arches=arches)
+
+
+@log_call
+def get_session():
+ """Get instance of Brew to talk with Koji"""
+ return Brew(global_config.koji_server)
+
+
+@log_call
+def get_download_dir():
+ """Return the directory holding all downloaded RPMs
+
+ If directory does not exist, it is created automatically.
+
+ :return: path to directory holding downloaded RPMs.
+ :rtype: str
+ """
+ download_dir = os.path.join(HOME_DIR, 'downloads')
+ if not os.path.exists(download_dir):
+ os.makedirs(download_dir)
+ return download_dir
+
+
+@log_call
+def download_rpm(url):
+ """Using curl to download a RPM from Koji
+
+ Currently, curl is called and runs in a spawned process. pycurl would be a
+ good way instead. This would be changed in the future.
+
+ :param str url: URL of a RPM to download.
+ :return: True if a RPM is downloaded successfully, False otherwise.
+ :rtype: bool
+ """
+ cmd = 'curl --silent -C - -O {} > {}'.format(
+ url, os.path.join(get_download_dir(),
+ os.path.basename(url)))
+ return_code = subprocess.call(cmd, shell=True)
+ if return_code > 0:
+ logger.error('curl fails with returned code: %d.', return_code)
+ return False
+ return True
+
+
+@log_call
+def download_rpms(rpms):
+ """Download RPMs
+
+ :param list rpms: list of RPMs to download.
+ """
+ def _download(rpm):
+ if rpm.is_downloaded:
+ logger.debug('Reuse %s', rpm.downloaded_file)
+ else:
+ logger.debug('Download %s', rpm.download_url)
+ download_rpm(rpm.download_url)
+
+ map(_download, rpms)
+
+
+@log_call
+def abipkgdiff(pkg_info1, pkg_info2):
+ """Run abipkgdiff against found two RPM packages
+
+ Construct and execute abipkgdiff to get ABI diff
+
+ abipkgdiff \
+ --d1 package1-debuginfo --d2 package2-debuginfo \
+ package1-rpm package2-rpm
+
+ Output to stdout or stderr from abipkgdiff is not captured. abipkgdiff is
+ called synchronously. fedabipkgdiff does not return until underlying
+ abipkgdiff finishes.
+
+ :param PkgInfo pkg_info1: the first package information provided for
+ abipkgdiff package1 paramter.
+ :param PkgInfo pkg_info2: the second package information provided for
+ abipkgdiff package2 paramter.
+ :return: return code of underlying abipkgdiff execution.
+ :rtype: int
+ """
+ cmd = 'abipkgdiff --d1 {0} --d2 {1} {2} {3}'.format(
+ pkg_info1.debuginfo_package.downloaded_file,
+ pkg_info2.debuginfo_package.downloaded_file,
+ pkg_info1.package.downloaded_file,
+ pkg_info2.package.downloaded_file)
+
+ if global_config.dry_run:
+ print 'DRY-RUN:', cmd
+ return
+
+ logger.debug('Run: %s', cmd)
+
+ print 'Comparing the ABI of binaries between {} and {}:'.format(
+ pkg_info1.package.filename, pkg_info2.package.filename)
+ print
+
+ proc = subprocess.Popen(shlex.split(cmd))
+ return proc.wait()
+
+
+def magic_construct(rpms):
+ """Construct RPMs into a magic structure
+
+ Convert list of
+
+ foo-1.0-1.fc22.i686
+ foo-debuginfo-1.0-1.fc22.i686
+ foo-devel-1.0-1.fc22.i686
+
+ to list of
+
+ (foo-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+ (foo-devel-1.0-1.fc22.i686, foo-debuginfo-1.0-1.fc22.i686)
+
+ :param rpms: a sequence of RPM packages.
+ :type rpms: list or tuple
+ :return: list of two-element tuple where the first element is a RPM package
+ and the second one is the debuginfo package.
+ :rtype: list
+ """
+ debuginfo = None
+ packages = []
+ for rpm in rpms:
+ if rpm.is_debuginfo:
+ debuginfo = rpm
+ else:
+ packages.append(rpm)
+ return [PkgInfo(package, debuginfo) for package in packages]
+
+
+@log_call
+def run_abipkgdiff(pkg1_infos, pkg2_infos):
+ """Run abipkgdiff
+
+ If one of the executions finds ABI differences, the return code is the
+ return code from abipkgdiff.
+
+ :param dict pkg1_infos: a mapping from arch to list of RPMs
+ :return: exit code of the last non-zero returned from underlying abipkgdiff
+ :rtype: number
+ """
+ arches = pkg1_infos.keys()
+ arches.sort()
+
+ return_code = 0
+
+ for arch in arches:
+ pkg_infos = magic_construct(pkg1_infos[arch])
+
+ for pkg_info in pkg_infos:
+ rpms = pkg2_infos[arch]
+
+ package = [rpm for rpm in rpms
+ if rpm.name == pkg_info.package.name][0]
+ debuginfo = [rpm for rpm in rpms
+ if rpm.name == pkg_info.debuginfo_package.name][0]
+
+ ret = abipkgdiff(pkg_info,
+ PkgInfo(package=package,
+ debuginfo_package=debuginfo))
+ if ret > 0:
+ return_code = ret
+
+ return return_code
+
+
+@log_call
+def diff_local_rpm_with_latest_rpm_from_koji():
+ """Diff against local rpm and remove latest rpm
+
+ This operation handles a local rpm and debuginfo rpm and remote ones
+ located in remote Koji server, that has specific distro specificed by
+ argument --from.
+
+ 1/ Suppose the packager has just locally built a package named
+ foo-3.0.fc24.rpm. To compare the ABI of this locally build package with the
+ latest stable package from Fedora 23, one would do:
+
+ fedabipkgdiff --from fc23 ./foo-3.0.fc24.rpm
+ """
+
+ from_distro = global_config.from_distro
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ local_rpm_file = global_config.NVR[0]
+ if not os.path.exists(local_rpm_file):
+ raise ValueError('{0} does not exist.'.format(local_rpm_file))
+
+ local_rpm = LocalRPM(local_rpm_file)
+ local_debuginfo = local_rpm.find_debuginfo()
+ if local_debuginfo is None:
+ raise ValueError(
+ 'debuginfo rpm {0} does not exist.'.format(local_debuginfo))
+
+ rpms = session.get_latest_built_rpms(local_rpm.name,
+ from_distro,
+ arches=local_rpm.arch)
+ download_rpms(rpms)
+ pkg_infos = make_rpms_usable_for_abipkgdiff(rpms)
+
+ rpms = pkg_infos.values()[0]
+ package, debuginfo = sorted(rpms, key=lambda rpm: rpm.name)
+ return abipkgdiff(PkgInfo(package, debuginfo),
+ PkgInfo(local_rpm, local_debuginfo))
+
+
+@log_call
+def make_rpms_usable_for_abipkgdiff(rpms):
+ """Prepare package information structure for running abipkgdiff
+
+ So far, RPMs input to this method are queried from Koji and abipkgdiff will
+ run against these RPMs. For convenience, these RPMs should be restructured
+ into a mapping so that subsequent operations could easily find RPMs from
+ arch.
+
+ For example, input RPMs are
+
+ [RPM(arch='x86_64', name='httpd'),
+ RPM(arch='i686', name='httpd'),
+ RPM(arch='x86_64', name='httpd-devel'),
+ RPM(arch='i686', name='http-debuginfo'),
+ RPM(arch='x86_64', name='httpd-debuginfo'),
+ ]
+
+ it is converted into mapping
+
+ {
+ 'x86_64': [RPM(arch='x86_64', name='httpd'),
+ RPM(arch='x86_64', name='httpd-devel'),
+ RPM(arch='x86_64', name='httpd-debuginfo')],
+ 'i686': [RPM(arch='i686', name='httpd'),
+ RPM(arch='i686', name='http-debuginfo')],
+ }
+
+ The order RPMs in the mapping is unpredictable. So, if they must be in a
+ particular order, caller is responsible for this.
+
+ :param list rpms: a list of RPMs
+ :return: a mapping from an arch to corresponding list of RPMs
+ :rtype: dict
+ """
+ result = {}
+ rpms_iter = groupby(sorted(rpms, key=lambda rpm: rpm.arch),
+ key=lambda item: item.arch)
+ for arch, rpms in rpms_iter:
+ result[arch] = list(rpms)
+ return result
+
+
+@log_call
+def diff_latest_rpms_based_on_distros():
+ """abipkgdiff rpms based on two distros
+
+ 2/ Suppose the packager wants to see how the ABIs of the package foo
+ evolved between fedora 19 and fedora 22. She would thus type the command:
+
+ fedabipkgdiff --from fc19 --to fc22 foo
+ """
+
+ from_distro = global_config.from_distro
+ to_distro = global_config.to_distro
+
+ if not is_distro_valid(from_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(from_distro))
+
+ if not is_distro_valid(to_distro):
+ raise InvalidDistroError('Invalid distro {0}'.format(distro))
+
+ package_name = global_config.NVR[0]
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.from_distro)
+ download_rpms(rpms)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+
+ rpms = session.get_latest_built_rpms(package_name,
+ distro=global_config.to_distro)
+ download_rpms(rpms)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+@log_call
+def diff_two_nvras_from_koji():
+ """Diff two nvras from koji
+
+ The arch probably omits, that means febabipkgdiff will diff all arches. If
+ specificed, the specific arch will be handled.
+
+ 3/ Suppose the packager wants to compare the ABI of two packages designated
+ by their name and version. She would issue a command like this:
+
+ fedabipkgdiff foo-1.0.fc19 foo-3.0.fc24
+ fedabipkgdiff foo-1.0.fc19.i686 foo-1.0.fc24.i686
+ """
+ left_rpm = koji.parse_NVRA(global_config.NVR[0])
+ right_rpm = koji.parse_NVRA(global_config.NVR[1])
+
+ if is_distro_valid(left_rpm['arch']) and \
+ is_distro_valid(right_rpm['arch']):
+ nvr = koji.parse_NVR(global_config.NVR[0])
+ params1 = (nvr['name'], nvr['version'], nvr['release'], None)
+
+ nvr = koji.parse_NVR(global_config.NVR[1])
+ params2 = (nvr['name'], nvr['version'], nvr['release'], None)
+ else:
+ params1 = (left_rpm['name'],
+ left_rpm['version'],
+ left_rpm['release'],
+ left_rpm['arch'])
+ params2 = (right_rpm['name'],
+ right_rpm['version'],
+ right_rpm['release'],
+ right_rpm['arch'])
+
+ build_id = session.get_rpm_build_id(*params1)
+ rpms = session.select_rpms_from_a_build(
+ build_id, params1[0], arches=params1[3],
+ select_subpackages=global_config.check_all_subpackages)
+ download_rpms(rpms)
+ pkg1_infos = make_rpms_usable_for_abipkgdiff(rpms)
+
+ build_id = session.get_rpm_build_id(*params2)
+ rpms = session.select_rpms_from_a_build(
+ build_id, params2[0], arches=params2[3],
+ select_subpackages=global_config.check_all_subpackages)
+ download_rpms(rpms)
+ pkg2_infos = make_rpms_usable_for_abipkgdiff(rpms)
+
+ return run_abipkgdiff(pkg1_infos, pkg2_infos)
+
+
+def build_commandline_args_parser():
+ parser = argparse.ArgumentParser(
+ description='Run abipkgdiff against RPM packages from koji')
+
+ parser.add_argument(
+ 'NVR',
+ nargs='*',
+ help='RPM package N-V-R, N-V-R-A, N, or a local RPM '
+ 'file name with relative or absolute path.')
+ parser.add_argument(
+ '--dry-run',
+ required=False,
+ dest='dry_run',
+ action='store_true',
+ help='Don\'t actually run abipkgdiff. The commands that should be '
+ 'run will be sent to stdout.')
+ parser.add_argument(
+ '--from',
+ required=False,
+ metavar='DISTRO',
+ dest='from_distro',
+ help='baseline Fedora distro, for example, fc23')
+ parser.add_argument(
+ '--to',
+ required=False,
+ metavar='DISTRO',
+ dest='to_distro',
+ help='which Fedora distro to compare, for example, fc24')
+ parser.add_argument(
+ '-a',
+ '--all-subpackages',
+ required=False,
+ action='store_true',
+ dest='check_all_subpackages',
+ help='Check all subpackages instead of only the package specificed in '
+ 'command line.')
+ parser.add_argument(
+ '--debug',
+ required=False,
+ action='store_true',
+ dest='debug',
+ help='show debug output')
+ parser.add_argument(
+ '--traceback',
+ required=False,
+ action='store_true',
+ dest='show_traceback',
+ help='show traceback when there is an exception thrown.')
+ parser.add_argument(
+ '--server',
+ required=False,
+ metavar='URL',
+ dest='koji_server',
+ default=DEFAULT_KOJI_SERVER,
+ help='URL of koji XMLRPC service. Default is {0}'.format(
+ DEFAULT_KOJI_SERVER))
+ parser.add_argument(
+ '--topdir',
+ required=False,
+ metavar='URL',
+ dest='koji_topdir',
+ default=DEFAULT_KOJI_TOPDIR,
+ help='URL for RPM files access')
+
+ return parser
+
+
+def main():
+ parser = build_commandline_args_parser()
+
+ args = parser.parse_args()
+
+ global global_config
+ global_config = args
+
+ global pathinfo
+ pathinfo = koji.PathInfo(topdir=global_config.koji_topdir)
+
+ global session
+ session = get_session()
+
+ if global_config.debug:
+ logger.setLevel(logging.DEBUG)
+
+ logger.debug(args)
+
+ if global_config.from_distro and global_config.to_distro is None and \
+ global_config.NVR:
+ returncode = diff_local_rpm_with_latest_rpm_from_koji()
+
+ elif global_config.from_distro and global_config.to_distro and \
+ global_config.NVR:
+ returncode = diff_latest_rpms_based_on_distros()
+
+ elif global_config.from_distro is None and \
+ global_config.to_distro is None and len(global_config.NVR) > 1:
+ returncode = diff_two_nvras_from_koji()
+
+ else:
+ print >>sys.stderr, 'Unknown arguments. Please refer to --help.'
+ returncode = 1
+
+ return returncode
+
+
+if __name__ == '__main__':
+ try:
+ main()
+ except KeyboardInterrupt:
+ if global_config.debug:
+ logger.debug('Terminate by user')
+ else:
+ print >>sys.stderr, 'Terminate by user'
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(2)
+ except Exception as e:
+ if global_config.debug:
+ logger.debug(str(e))
+ else:
+ print >>sys.stderr, str(e)
+ if global_config.show_traceback:
+ raise
+ else:
+ sys.exit(1)
--
2.5.5
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2016-05-13 10:34 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-01 0:00 a new tool fedabipkgdiff Chenxiong Qi
2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Chenxiong Qi
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Dodji Seketeli
2016-01-01 0:00 ` Fix adding fedabipkgdiff to source distribution tarball Dodji Seketeli
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).