From: "Serhei Makarov" <me@serhei.io>
To: "Keith Seitz" <keiths@redhat.com>, Bunsen <bunsen@sourceware.org>
Subject: Re: [RFC PATCH] New script `summarize'
Date: Fri, 25 Sep 2020 11:07:10 -0400 [thread overview]
Message-ID: <23de1a72-0feb-4937-95a3-07d185e44b02@www.fastmail.com> (raw)
In-Reply-To: <20200924185424.1496842-1-keiths@redhat.com>
> diff --git a/bunsen.py b/bunsen.py
> index b089d80..99c4cc0 100755
> --- a/bunsen.py
> +++ b/bunsen.py
> @@ -622,6 +622,34 @@ class Testrun(dict):
> # XXX Set summary=False if JSON was missing testcases.
> self.summary = self.summary and 'testcases' in json_data
>
> + # Return properties of this Testrun as printable strings, or
Suggest "Return configuration properties"
for consistency with other code/comments that use the term 'configuration'
for these properties.
> + # "<unknown PROPERTY>" if unknown. Returns a dictionary containing
> + # keys for architecture, board, branch, version.
> +
> + def get_info_strings(self):
> + info = dict()
> + if 'arch' in self:
> + info['architecture'] = self.arch
> + else:
> + info['architecture'] = '<unknown arch>'
> +
> + if 'board' in self:
> + info['board'] = self.board
> + else:
> + info['board'] = '<unknown board>'
> +
> + if 'source_branch' in self:
> + info['branch'] = self.source_branch
> + else:
> + info['branch'] = '<unknown branch>'
> +
> + if 'version' in self:
> + info['version'] = self.version
> + else:
> + info['version'] = '<unknown version>'
> +
> + return info
> +
Looks OK to me, I may simplify or generalize this code with a later patch.
(e.g. by taking a dictionary of field name -> display name as an optional argument,
or having that dictionary predefined somewhere and defaulting to return
all the keys/values that are *not* specific to the Bunsen storage (such as bunsen_commit_id)
-- needs some thinking on my part, so please commit as-is for now).
It's worth noting that the set of configuration keys will vary by project,
e.g. for SystemTap 'osver' (the Linux distribution) is a more important field.
> diff --git a/scripts-master/summarize.py b/scripts-master/summarize.py
> new file mode 100755
> index 0000000..e46db18
> --- /dev/null
> +++ b/scripts-master/summarize.py
> @@ -0,0 +1,68 @@
> +#!/usr/bin/env python3
> +
> +# Display a DejaGNU-like test summary, given the bunsen commit
> +# of the desired test run. Optionally also takes a comma-separated
> +# list of glob expressions to limit results.
> +
> +info = "summarize.py <bunsen_commit> [tests]"
> +cmdline_args = [
> + ('commit', None, '<bunsen_commit>',
> + "commit to fetch results for"),
> + ('tests', None, '<test_globs>',
> + "comma-separated list of glob expressions of tests to summarize")
> +]
> +
> +import sys
> +import bunsen
> +from collections import Counter
> +from pathlib import PurePath
> +
> +# A list of test outcomes in output order.
> +outcome_labels = {
> + 'PASS' : 'expected passes',
> + 'FAIL' : 'unexpected failures',
> + 'XPASS' : 'unexpected successes',
> + 'XFAIL' : 'expected failures',
> + 'KPASS' : 'unknown successes',
> + 'KFAIL' : 'known failures',
> + 'UNTESTED' : 'untested testcases',
> + 'UNRESOLVED' : 'unresolved testcases',
> + 'UNSUPPORTED' : 'unsupported tests',
> + 'ERROR' : 'errors',
> + 'WARNING' : 'warnings'
> +}
> +
> +if __name__ == '__main__':
> + b = bunsen.Bunsen()
> + opts = b.cmdline_args(sys.argv, info=info, args=cmdline_args,
> + required_args=['commit'],
> optional_args=['tests'])
> +
> + testrun = b.testrun(opts.commit)
> + all_tests = testrun.testcases
> + found_tests = []
> + if opts.tests is not None:
> + for glob in opts.tests.split(','):
> + found_tests.extend([t for t in all_tests if
> PurePath(t['name']).match(glob)])
> + else:
> + found_tests = all_tests
> +
> + if found_tests:
> + info = testrun.get_info_strings()
> +
> + project = b.tags[0] if len(b.tags) == 1 else '<multiple
> projects>'
> + print(f'Summary for commit {opts.commit} of {project} version
> {info["version"]}')
> + print(f'from branch {info["branch"]} on {info["architecture"]}
> using {info["board"]}')
> + if opts.tests is not None:
> + print(f'limiting results to tests matching: {opts.tests}')
> + print()
> +
> + # Collate results for outcomes
> + c = Counter(t['outcome'] for t in found_tests)
> +
> + # We could simply loop over the keys of the Counter, but that
> would not necessarily give
> + # us the same output order as DejaGNU itself.
> + for l in outcome_labels:
> + if c[l] != 0:
> + print('# of %-26s %d' % (outcome_labels[l], c[l]))
> + else:
> + print(f'found no tests matching \"{opts.tests}\"')
LGTM.
next prev parent reply other threads:[~2020-09-25 15:07 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-24 18:54 Keith Seitz
2020-09-25 15:07 ` Serhei Makarov [this message]
2020-09-25 17:02 ` Keith Seitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=23de1a72-0feb-4937-95a3-07d185e44b02@www.fastmail.com \
--to=me@serhei.io \
--cc=bunsen@sourceware.org \
--cc=keiths@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).