From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by sourceware.org (Postfix) with ESMTPS id AF772394443E for ; Fri, 25 Sep 2020 15:07:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org AF772394443E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=serhei.io Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=me@serhei.io Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id D71E045A; Fri, 25 Sep 2020 11:07:31 -0400 (EDT) Received: from imap21 ([10.202.2.71]) by compute7.internal (MEProxy); Fri, 25 Sep 2020 11:07:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=serhei.io; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=1PEYDxPk/B3VrREIhoVTjwgMQqwgEnG 3SRvrukopjh4=; b=qA5YzdsYUa8V8XkJwcGOwZ6PmjuCQtPoR3F6hkvWWce+74t bmQlTPXWwk6qbO49lmW0ANyNQVO72YzvqTxwOKVTqrl9hLS+HmaPFQkhPzvrYFBe bsmFsnVx2W9u46ZksMA7x3sCYuau6Wp5wL1S5L8Ou3eqUHXzQ1VHQmcy+ST9IChy eZEM9Zw9EyuKZ77R46Sb5+9J5/89SPpX82ADwVZ091lKJXCWyDfBZVstzN1iuIZB iB0e/FUU7btKcdftclaIKdwxImHB7As6g9bHvAh5kVWwYRE++myJvt21HDIgL1KV jNRWkiHGsjN9FJsVmLBY8IPvxlx+b5nI3cq79Rg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=1PEYDx Pk/B3VrREIhoVTjwgMQqwgEnG3SRvrukopjh4=; b=DlYEtzh7iLfvb23Oj3OJcC M0lrFJLRH1dMJv8sx+sWPaMWf/l2amdX9pkFEZv/qwOwX1VLLiKTMrds+eINOfE5 kTk6K3mOJxa9Ax+k31GcazSRRm0hLKS53rxNbq1srIdYeCkbcS31MVBOHyaAc4EG ibrgInPFDiewS/X95Y5jdX6Pv0nIxHdztEfmgoH/s3VSGvnrkenR4FHKadqrOyAi 0yZqj53Zaqhcj7v82faJ4zAey4ZlKbSgnLIgIOH0nFmnYzu9mhJPoZynnR3GUFmm HtpqQN5X02dVMNmryx9Ww2i/g+KjvtUjxahF7/yOfhc2G/TbowTtF6ZMJairBc9A == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvddtgdekfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedfufgvrhhh vghiucforghkrghrohhvfdcuoehmvgesshgvrhhhvghirdhioheqnecuggftrfgrthhtvg hrnhepfeejgffggeffudehiedtledtgeejtdekgfeuleehuddtgfegvdfghfejkeeigfdt necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgvse hsvghrhhgvihdrihho X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 1D224660069; Fri, 25 Sep 2020 11:07:23 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-355-g3ece53b-fm-20200922.004-g3ece53b9 Mime-Version: 1.0 Message-Id: <23de1a72-0feb-4937-95a3-07d185e44b02@www.fastmail.com> In-Reply-To: <20200924185424.1496842-1-keiths@redhat.com> References: <20200924185424.1496842-1-keiths@redhat.com> Date: Fri, 25 Sep 2020 11:07:10 -0400 From: "Serhei Makarov" To: "Keith Seitz" , Bunsen Subject: Re: [RFC PATCH] New script `summarize' Content-Type: text/plain X-Spam-Status: No, score=-9.2 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, GIT_PATCH_0, JMQ_SPF_NEUTRAL, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: bunsen@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Bunsen mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Sep 2020 15:07:36 -0000 > diff --git a/bunsen.py b/bunsen.py > index b089d80..99c4cc0 100755 > --- a/bunsen.py > +++ b/bunsen.py > @@ -622,6 +622,34 @@ class Testrun(dict): > # XXX Set summary=False if JSON was missing testcases. > self.summary = self.summary and 'testcases' in json_data > > + # Return properties of this Testrun as printable strings, or Suggest "Return configuration properties" for consistency with other code/comments that use the term 'configuration' for these properties. > + # "" if unknown. Returns a dictionary containing > + # keys for architecture, board, branch, version. > + > + def get_info_strings(self): > + info = dict() > + if 'arch' in self: > + info['architecture'] = self.arch > + else: > + info['architecture'] = '' > + > + if 'board' in self: > + info['board'] = self.board > + else: > + info['board'] = '' > + > + if 'source_branch' in self: > + info['branch'] = self.source_branch > + else: > + info['branch'] = '' > + > + if 'version' in self: > + info['version'] = self.version > + else: > + info['version'] = '' > + > + return info > + Looks OK to me, I may simplify or generalize this code with a later patch. (e.g. by taking a dictionary of field name -> display name as an optional argument, or having that dictionary predefined somewhere and defaulting to return all the keys/values that are *not* specific to the Bunsen storage (such as bunsen_commit_id) -- needs some thinking on my part, so please commit as-is for now). It's worth noting that the set of configuration keys will vary by project, e.g. for SystemTap 'osver' (the Linux distribution) is a more important field. > diff --git a/scripts-master/summarize.py b/scripts-master/summarize.py > new file mode 100755 > index 0000000..e46db18 > --- /dev/null > +++ b/scripts-master/summarize.py > @@ -0,0 +1,68 @@ > +#!/usr/bin/env python3 > + > +# Display a DejaGNU-like test summary, given the bunsen commit > +# of the desired test run. Optionally also takes a comma-separated > +# list of glob expressions to limit results. > + > +info = "summarize.py [tests]" > +cmdline_args = [ > + ('commit', None, '', > + "commit to fetch results for"), > + ('tests', None, '', > + "comma-separated list of glob expressions of tests to summarize") > +] > + > +import sys > +import bunsen > +from collections import Counter > +from pathlib import PurePath > + > +# A list of test outcomes in output order. > +outcome_labels = { > + 'PASS' : 'expected passes', > + 'FAIL' : 'unexpected failures', > + 'XPASS' : 'unexpected successes', > + 'XFAIL' : 'expected failures', > + 'KPASS' : 'unknown successes', > + 'KFAIL' : 'known failures', > + 'UNTESTED' : 'untested testcases', > + 'UNRESOLVED' : 'unresolved testcases', > + 'UNSUPPORTED' : 'unsupported tests', > + 'ERROR' : 'errors', > + 'WARNING' : 'warnings' > +} > + > +if __name__ == '__main__': > + b = bunsen.Bunsen() > + opts = b.cmdline_args(sys.argv, info=info, args=cmdline_args, > + required_args=['commit'], > optional_args=['tests']) > + > + testrun = b.testrun(opts.commit) > + all_tests = testrun.testcases > + found_tests = [] > + if opts.tests is not None: > + for glob in opts.tests.split(','): > + found_tests.extend([t for t in all_tests if > PurePath(t['name']).match(glob)]) > + else: > + found_tests = all_tests > + > + if found_tests: > + info = testrun.get_info_strings() > + > + project = b.tags[0] if len(b.tags) == 1 else ' projects>' > + print(f'Summary for commit {opts.commit} of {project} version > {info["version"]}') > + print(f'from branch {info["branch"]} on {info["architecture"]} > using {info["board"]}') > + if opts.tests is not None: > + print(f'limiting results to tests matching: {opts.tests}') > + print() > + > + # Collate results for outcomes > + c = Counter(t['outcome'] for t in found_tests) > + > + # We could simply loop over the keys of the Counter, but that > would not necessarily give > + # us the same output order as DejaGNU itself. > + for l in outcome_labels: > + if c[l] != 0: > + print('# of %-26s %d' % (outcome_labels[l], c[l])) > + else: > + print(f'found no tests matching \"{opts.tests}\"') LGTM.