From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 91183 invoked by alias); 30 Jan 2018 13:55:54 -0000 Mailing-List: contact libabigail-help@sourceware.org; run by ezmlm Precedence: bulk List-Post: List-Help: List-Id: List-Subscribe: Sender: libabigail-owner@sourceware.org Received: (qmail 91085 invoked by uid 89); 30 Jan 2018 13:55:53 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Checked: by ClamAV 0.99.2 on sourceware.org X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.4 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KAM_LAZY_DOMAIN_SECURITY,KAM_NUMSUBJECT,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 spammy=rpm, UD:call, RPM, sk:require X-Spam-Status: No, score=-25.4 required=5.0 tests=BAYES_00,GIT_PATCH_0,GIT_PATCH_1,GIT_PATCH_2,GIT_PATCH_3,KAM_LAZY_DOMAIN_SECURITY,KAM_NUMSUBJECT,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on sourceware.org X-Spam-Level: X-HELO: mail-pf0-f171.google.com Received: from mail-pf0-f171.google.com (HELO mail-pf0-f171.google.com) (209.85.192.171) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 30 Jan 2018 13:55:46 +0000 Received: by mail-pf0-f171.google.com with SMTP id 17so7574627pfw.11 for ; Tue, 30 Jan 2018 05:55:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=sCBVw7cuwVannH7HQcFYS0qxe39t+w2/XSCcFXGuR+M=; b=Lgh3AnlDiUUuTcrqOu9G4uOyrMzkQl3X8zhjwVGLFQXQN/nBKPmTgvJ+xJfRbnPRpC Lnd6+qlzPUeBfmHdEwmMLu+1jeHotiIn+6OhKwdCay0WLY4pnJLVOs1aQ3dfQDdUI0Ua OA7W8lc91EnOjVQYw61Y9zVX5jHI3Ztj5ZmMPeMKgf9A+Fw5dW8FLa5log8XkEMC9MDN IOEmBGtoR8TCaQQ0iHeb2cZ3oRfMK8txVlomnqjZTPdBRS4v+fMbFgbEpuixkn1t7zua N+Kc4fuKS/zZmxw0Cr3GoYoUHMcLXGG27xfdUCmvaYmR7UdIY3ftcbmDGIFSpVl3WdyU hUwg== X-Gm-Message-State: AKwxytcmrs0PVoLl++ennl03knxdDnZzcMNVgi9xTqSTtkJrknBbDQoI g/FzJTusgBswSc7LDOAaIMhro2CVWdw= X-Google-Smtp-Source: AH8x2263SReGWvZSslrZbEVf7DKnhiglZPt7lck8oD6luvQUfKhgGxiaOlh8VMIp8GUaLygdQhkQmw== X-Received: by 10.98.72.19 with SMTP id v19mr30817233pfa.107.1517320544254; Tue, 30 Jan 2018 05:55:44 -0800 (PST) Received: from cqi-laptop.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id z10sm22607421pgs.17.2018.01.30.05.55.42 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 30 Jan 2018 05:55:43 -0800 (PST) From: cqi@redhat.com To: libabigail@sourceware.org Cc: Chenxiong Qi Subject: [PATCH] Bug 22722 - Make fedabidiff and its tests support both python 3 and 2 Date: Mon, 01 Jan 2018 00:00:00 -0000 Message-Id: <20180130135529.13076-1-cqi@redhat.com> X-Mailer: git-send-email 2.14.3 X-IsSubscribed: yes X-SW-Source: 2018-q1/txt/msg00010.txt.bz2 From: Chenxiong Qi This patch makes fedabipkgdiff Python 3 compatible. All tests written in Python are updated and compatible with Python 3 as well. Tests run with Python 3 by default. It could be disabled by a new option --disable-py3-tests if it is unnecessary. * configure.ac: Add new option --disable-py3-tests. Add new test runner file tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiff-py3. Add required six Python module. * tests/Makefile.am: Add new test files tests/runtestdefaultsupprs-py3 and tests/runtestfedabipkgdiff-py3 according to the new option --disable-py3-tests. * tests/mockfedabipkgdiff.in: Convert print statement to six.print_. * tests/runtestdefaultsupprs-py3.in: New shell script to run test runtestdefaultsupprs with Python 3. * tests/runtestdefaultsupprs.py.in: Remove trailing semicolons. Repalce tabs with proper number of spaces. * tests/runtestfedabipkgdiff-py3.in: New shell script to run test runtestfedabipkgdiff with Python 3. * tests/runtestfedabipkgdiff.py.in: Use python from env in shebang instead of a fixed path to a Python interpreter. * tests/update-test-output.py: Likewise. And convert print statement to the print function call which is available after importing print_function from __future__ module. * tools/fedabipkgdiff: Globally replace print statement with a function call to print which is available by importing print_function from __future__ module. Use six.print_ to output string to stderr instead. Convert function call to map to for-loop. Signed-off-by: Chenxiong Qi --- configure.ac | 20 ++++++++++++- tests/Makefile.am | 14 +++++++++ tests/mockfedabipkgdiff.in | 19 ++++++------ tests/runtestdefaultsupprs-py3.in | 2 ++ tests/runtestdefaultsupprs.py.in | 62 ++++++++++++++++++++------------------- tests/runtestfedabipkgdiff-py3.in | 2 ++ tests/runtestfedabipkgdiff.py.in | 2 +- tests/update-test-output.py | 8 +++-- tools/fedabipkgdiff | 33 ++++++++++++--------- 9 files changed, 104 insertions(+), 58 deletions(-) create mode 100644 tests/runtestdefaultsupprs-py3.in create mode 100644 tests/runtestfedabipkgdiff-py3.in diff --git a/configure.ac b/configure.ac index 4963ee5..9bc92f2 100644 --- a/configure.ac +++ b/configure.ac @@ -97,6 +97,12 @@ AC_ARG_ENABLE([fedabipkgdiff], ENABLE_FEDABIPKGDIFF=$enableval, ENABLE_FEDABIPKGDIFF=auto) +AC_ARG_ENABLE([py3-tests], + AS_HELP_STRING([--disable-py3-tests=yes|no|auto], + [disable running tests with Python 3]), + DISABLE_PY3_TESTS=$enableval, + DISABLE_PY3_TESTS=auto) + dnl ************************************************* dnl check for dependencies dnl ************************************************* @@ -351,7 +357,7 @@ if test x$CHECK_DEPS_FOR_FEDABIPKGDIFF = xyes; then REQUIRED_PYTHON_MODULES_FOR_FEDABIPKGDIFF="\ argparse logging os re subprocess sys urlparse \ - xdg koji mock rpm imp tempfile mimetypes shutil" + xdg koji mock rpm imp tempfile mimetypes shutil six" if test x$ENABLE_FEDABIPKGDIFF != xno; then AX_CHECK_PYTHON_MODULES([$REQUIRED_PYTHON_MODULES_FOR_FEDABIPKGDIFF], @@ -399,6 +405,14 @@ fi AM_CONDITIONAL(ENABLE_FEDABIPKGDIFF, test x$ENABLE_FEDABIPKGDIFF = xyes) + +AC_MSG_NOTICE(Run tests with Python 3... $DISABLE_PY3_TESTS) +if test x$DISABLE_PY3_TESTS = xauto -o x$DISABLE_PY3_TESTS = xyes; then + ENABLE_PY3_TESTS=yes +fi + +AM_CONDITIONAL(ENABLE_PY3_TESTS, test x$ENABLE_PY3_TESTS = xyes) + dnl Check for dependency: libzip LIBZIP_VERSION=0.10.1 @@ -556,8 +570,12 @@ AC_CONFIG_FILES([tests/mockfedabipkgdiff], [chmod +x tests/mockfedabipkgdiff]) AC_CONFIG_FILES([tests/runtestfedabipkgdiff.py], [chmod +x tests/runtestfedabipkgdiff.py]) +AC_CONFIG_FILES([tests/runtestfedabipkgdiff-py3], + [chmod +x tests/runtestfedabipkgdiff-py3]) AC_CONFIG_FILES([tests/runtestdefaultsupprs.py], [chmod +x tests/runtestdefaultsupprs.py]) +AC_CONFIG_FILES([tests/runtestdefaultsupprs-py3], + [chmod +x tests/runtestdefaultsupprs-py3]) AC_OUTPUT diff --git a/tests/Makefile.am b/tests/Makefile.am index 0a7f4b9..9b31e98 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -20,6 +20,9 @@ FEDABIPKGDIFF_TEST = if ENABLE_FEDABIPKGDIFF FEDABIPKGDIFF_TEST += runtestfedabipkgdiff.py endif +if ENABLE_PY3_TESTS +FEDABIPKGDIFF_TEST += runtestfedabipkgdiff-py3 +endif TESTS= \ $(FEDABIPKGDIFF_TEST) \ @@ -43,10 +46,15 @@ runtestabidiffexit \ runtestdefaultsupprs.py \ $(CXX11_TESTS) +if ENABLE_PY3_TESTS +TESTS += runtestdefaultsupprs-py3 +endif + EXTRA_DIST = \ runtestcanonicalizetypes.sh.in \ runtestfedabipkgdiff.py.in \ +runtestfedabipkgdiff-py3.in \ mockfedabipkgdiff.in \ test-valgrind-suppressions.supp @@ -136,9 +144,15 @@ runtestcanonicalizetypes.sh$(EXEEXT): runtestfedabipkgdiff_py_SOURCES = runtestfedabipkgdiff.py$(EXEEXT): +runtestfedabipkgdiff_py3_SOURCES = +runtestfedabipkgdiff-py3$(EXEEXT): + runtestdefaultsupprs_py_SOURCES = runtestdefaultsupprs.py$(EXEEXT): +runtestdefaultsupprs_py3_SOURCES = +runtestdefaultsupprs-py3$(EXEEXT): + AM_CPPFLAGS=-I${abs_top_srcdir}/include \ -I${abs_top_builddir}/include -I${abs_top_srcdir}/tools -fPIC diff --git a/tests/mockfedabipkgdiff.in b/tests/mockfedabipkgdiff.in index aac99d1..1168b92 100644 --- a/tests/mockfedabipkgdiff.in +++ b/tests/mockfedabipkgdiff.in @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env python # -*- coding: utf-8 -*- # -*- Mode: Python # @@ -55,13 +55,14 @@ variables. import os import tempfile import imp +import six try: from mock import patch except ImportError: import sys - print >>sys.stderr, \ - 'mock is required to run tests. Please install before running tests.' + six.print_('mock is required to run tests. Please install before running ' + 'tests.', file=sys.stderr) sys.exit(1) ABIPKGDIFF = '@abs_top_builddir@/tools/abipkgdiff' @@ -246,29 +247,29 @@ rpms = [ }, # RPMs of build vte291-0.39.1-1.fc22 {'build_id': 600011, - 'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22', + 'name': 'vte291', 'version': '0.39.1', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22', }, {'build_id': 600011, - 'name': 'vte291-devel', 'version': '0.39.1', 'release': '1.fc22', + 'name': 'vte291-devel', 'version': '0.39.1', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22', }, {'build_id': 600011, - 'name': 'vte291-debuginfo', 'version': '0.39.1', 'release': '1.fc22', + 'name': 'vte291-debuginfo', 'version': '0.39.1', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.1-1.fc22', }, # RPMs of build vte291-0.39.90-1.fc22 {'build_id': 612610, - 'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22', + 'name': 'vte291', 'version': '0.39.90', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22', }, {'build_id': 612610, - 'name': 'vte291-devel', 'version': '0.39.90', 'release': '1.fc22', + 'name': 'vte291-devel', 'version': '0.39.90', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22', }, {'build_id': 612610, - 'name': 'vte291-debuginfo', 'version': '0.39.90', 'release': '1.fc22', + 'name': 'vte291-debuginfo', 'version': '0.39.90', 'release': '1.fc22', 'arch': 'x86_64', 'nvr': 'vte291-0.39.90-1.fc22', }, ] diff --git a/tests/runtestdefaultsupprs-py3.in b/tests/runtestdefaultsupprs-py3.in new file mode 100644 index 0000000..7b856d0 --- /dev/null +++ b/tests/runtestdefaultsupprs-py3.in @@ -0,0 +1,2 @@ +#!/bin/bash +python3 "@abs_top_builddir@/tests/runtestdefaultsupprs.py" \ No newline at end of file diff --git a/tests/runtestdefaultsupprs.py.in b/tests/runtestdefaultsupprs.py.in index 41b0b28..9feafe7 100644 --- a/tests/runtestdefaultsupprs.py.in +++ b/tests/runtestdefaultsupprs.py.in @@ -67,7 +67,8 @@ def ensure_output_dir_created(): pass if not os.path.exists(output_dir): - sys.exit(1); + sys.exit(1) + def run_abidiff_tests(): """Run the abidiff default suppression tests. @@ -90,9 +91,9 @@ def run_abidiff_tests(): default_suppression = output_dir + "/default.abignore" with open(default_suppression, 'w') as out: - out.write('\n'); + out.write('\n') - result = 0; + result = 0 for test_spec in abidiff_test_specs: binary1 = test_spec[0] binary2 = test_spec[1] @@ -111,40 +112,40 @@ def run_abidiff_tests(): # The environment variables that say where to find the default # suppression specifications loaded by libabigail. envs = { - 'LIBABIGAIL_DEFAULT_SYSTEM_SUPPRESSION_FILE' : default_suppression, - 'LIBABIGAIL_DEFAULT_USER_SUPPRESSION_FILE' : default_suppression + 'LIBABIGAIL_DEFAULT_SYSTEM_SUPPRESSION_FILE': default_suppression, + 'LIBABIGAIL_DEFAULT_USER_SUPPRESSION_FILE': default_suppression } # Initialize the environment variables above to their default # value. for name, value in envs.items(): - os.environ[name] = value; + os.environ[name] = value for env_name in envs: env_vars = os.environ if suppressions: env_vars[env_name] = suppressions - with open(output_path, 'w') as out_file: - subprocess.call(cmd, env=env_vars, stdout=out_file) + with open(output_path, 'w') as out_file: + subprocess.call(cmd, env=env_vars, stdout=out_file) - diffcmd = ['diff', '-u', reference_report_path, - output_path] + diffcmd = ['diff', '-u', reference_report_path, output_path] - return_code = subprocess.call(diffcmd) + return_code = subprocess.call(diffcmd) if return_code: result = return_code sys.stderr.write("failed abidiff test " - "for env var '" + e + "'\n"); + "for env var '" + e + "'\n") - del env_vars[env_name]; + del env_vars[env_name] try: os.remove(default_suppression) except: pass - return result; + return result + def run_abipkgdiff_tests(): """Run the abipkgdiff default suppression tests. @@ -167,9 +168,9 @@ def run_abipkgdiff_tests(): default_suppression = output_dir + "/default.abignore" with open(default_suppression, 'w') as out: - out.write('\n'); + out.write('\n') - result = 0; + result = 0 for test_spec in abipkgdiff_test_specs: pkg1 = test_spec[0] pkg2 = test_spec[1] @@ -188,40 +189,40 @@ def run_abipkgdiff_tests(): # The environment variables that say where to find the default # suppression specifications loaded by libabigail. envs = { - 'LIBABIGAIL_DEFAULT_SYSTEM_SUPPRESSION_FILE' : default_suppression, - 'LIBABIGAIL_DEFAULT_USER_SUPPRESSION_FILE' : default_suppression + 'LIBABIGAIL_DEFAULT_SYSTEM_SUPPRESSION_FILE': default_suppression, + 'LIBABIGAIL_DEFAULT_USER_SUPPRESSION_FILE': default_suppression } # Initialize the environment variables above to their default # value. for name, value in envs.items(): - os.environ[name] = value; + os.environ[name] = value for env_name in envs: env_vars = os.environ if suppressions: env_vars[env_name] = suppressions - with open(output_path, 'w') as out_file: - subprocess.call(cmd, env=env_vars, stdout=out_file) + with open(output_path, 'w') as out_file: + subprocess.call(cmd, env=env_vars, stdout=out_file) - diffcmd = ['diff', '-u', reference_report_path, - output_path] + diffcmd = ['diff', '-u', reference_report_path, output_path] - return_code = subprocess.call(diffcmd) + return_code = subprocess.call(diffcmd) if return_code: result = return_code sys.stderr.write("failed abipkgdiff test " - "for env var '" + e + "'\n"); + "for env var '" + e + "'\n") - del env_vars[env_name]; + del env_vars[env_name] try: os.remove(default_suppression) except: pass - return result; + return result + def main(): """The main entry point of this program. @@ -235,12 +236,13 @@ def main(): """ ensure_output_dir_created() - result = 0; + result = 0 result = run_abidiff_tests() if result: - return result; + return result result = run_abipkgdiff_tests() - return result; + return result + if __name__ == '__main__': exit_code = main() diff --git a/tests/runtestfedabipkgdiff-py3.in b/tests/runtestfedabipkgdiff-py3.in new file mode 100644 index 0000000..b4cbe78 --- /dev/null +++ b/tests/runtestfedabipkgdiff-py3.in @@ -0,0 +1,2 @@ +#!/bin/bash +python3 "@abs_top_builddir@/tests/runtestfedabipkgdiff.py" \ No newline at end of file diff --git a/tests/runtestfedabipkgdiff.py.in b/tests/runtestfedabipkgdiff.py.in index 78898f3..04429b6 100755 --- a/tests/runtestfedabipkgdiff.py.in +++ b/tests/runtestfedabipkgdiff.py.in @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/env python # -*- coding: utf-8 -*- # -*- Mode: Python # diff --git a/tests/update-test-output.py b/tests/update-test-output.py index 4017dd0..84afd4e 100755 --- a/tests/update-test-output.py +++ b/tests/update-test-output.py @@ -1,14 +1,16 @@ -#!/bin/python +#!/usr/bin/env python # This program generates the copy commands you should use to update # the reference data for tests /tests/runtest* that emit an # output that is compared against a reference output. -# +# # It takes in argument the diff result emitted by # /tests/runtest*, and emits on standard output a series of # 'cp ' commands to execute to update reference data of # the test. +from __future__ import print_function + import fileinput import re import sys @@ -31,7 +33,7 @@ def main(): try: opts, args = getopt.getopt(sys.argv[1:], "hi", ["help"]) except getopt.GetoptError as err: - print str(err) + print(str(err)) display_usage() for opt, arg in opts: diff --git a/tools/fedabipkgdiff b/tools/fedabipkgdiff index da303ad..5500104 100755 --- a/tools/fedabipkgdiff +++ b/tools/fedabipkgdiff @@ -22,6 +22,8 @@ # # Author: Chenxiong Qi +from __future__ import print_function + import argparse import glob import logging @@ -39,6 +41,7 @@ import xdg.BaseDirectory import rpm import koji +import six # @file # @@ -461,7 +464,8 @@ class RPMCollection(object): self.ancillary_rpms = {} if rpms: - map(self.add, rpms) + for rpm in rpms: + self.add(rpm) @classmethod def gather_from_dir(cls, rpm_file, all_rpms=None): @@ -561,7 +565,7 @@ class RPMCollection(object): if r.is_debuginfo: result.append(r) return result - + def generate_comparison_halves(rpm_col1, rpm_col2): """Iterate RPM collection and peer's to generate comparison halves""" @@ -974,7 +978,7 @@ def download_rpm(url): url, os.path.join(get_download_dir(), os.path.basename(url))) if global_config.dry_run: - print 'DRY-RUN:', cmd + print('DRY-RUN:', cmd) return return_code = subprocess.call(cmd, shell=True) @@ -997,7 +1001,8 @@ def download_rpms(rpms): logger.debug('Download %s', rpm.download_url) download_rpm(rpm.download_url) - map(_download, rpms) + for rpm in rpms: + _download(rpm) @log_call @@ -1019,7 +1024,7 @@ def build_path_to_abipkgdiff(): def format_debug_info_pkg_options(option, debuginfo_list): """Given a list of debug info package descriptors return an option string that looks like: - + option dbg.rpm1 option dbgrpm2 ... :param: list debuginfo_list a list of instances of the RPM class @@ -1124,14 +1129,14 @@ def abipkgdiff(cmp_half1, cmp_half2): cmd = filter(lambda s: s != '', cmd) if global_config.dry_run: - print 'DRY-RUN:', ' '.join(cmd) + print('DRY-RUN:', ' '.join(cmd)) return logger.debug('Run: %s', ' '.join(cmd)) - print 'Comparing the ABI of binaries between {0} and {1}:'.format( - cmp_half1.subject.filename, cmp_half2.subject.filename) - print + print('Comparing the ABI of binaries between {0} and {1}:'.format( + cmp_half1.subject.filename, cmp_half2.subject.filename)) + print() proc = subprocess.Popen(' '.join(cmd), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) @@ -1160,9 +1165,9 @@ def abipkgdiff(cmp_half1, cmp_half2): has_abi_change = proc.returncode & ABIDIFF_ABI_CHANGE if is_internal_error: - print >>sys.stderr, stderr + six.print_(stderr, file=sys.stderr) elif is_ok or has_abi_change: - print stdout + print(stdout) return proc.returncode @@ -1555,7 +1560,7 @@ def main(): if both_nvr or both_nvra: return diff_two_nvras_from_koji() - print >>sys.stderr, 'Unknown arguments. Please refer to --help.' + six.print_('Unknown arguments. Please refer to --help.', file=sys.stderr) return 1 @@ -1568,7 +1573,7 @@ if __name__ == '__main__': if global_config.debug: logger.debug('Terminate by user') else: - print >>sys.stderr, 'Terminate by user' + six.print_('Terminate by user', file=sys.stderr) if global_config.show_traceback: raise else: @@ -1579,7 +1584,7 @@ if __name__ == '__main__': if global_config.debug: logger.debug(str(e)) else: - print >>sys.stderr, str(e) + six.print_(str(e), file=sys.stderr) if global_config.show_traceback: raise else: -- 2.14.3