public inbox for bunsen@sourceware.org
 help / color / mirror / Atom feed
From: "Serhei Makarov" <me@serhei.io>
To: "Keith Seitz" <keiths@redhat.com>, Bunsen <bunsen@sourceware.org>
Subject: Re: [RFC PATCH] New script `summarize'
Date: Fri, 25 Sep 2020 11:07:10 -0400	[thread overview]
Message-ID: <23de1a72-0feb-4937-95a3-07d185e44b02@www.fastmail.com> (raw)
In-Reply-To: <20200924185424.1496842-1-keiths@redhat.com>

> diff --git a/bunsen.py b/bunsen.py
> index b089d80..99c4cc0 100755
> --- a/bunsen.py
> +++ b/bunsen.py
> @@ -622,6 +622,34 @@ class Testrun(dict):
>              # XXX Set summary=False if JSON was missing testcases.
>              self.summary = self.summary and 'testcases' in json_data
>  
> +    # Return properties of this Testrun as printable strings, or
Suggest "Return configuration properties"
for consistency with other code/comments that use the term 'configuration'
for these properties.

> +    # "<unknown PROPERTY>" if unknown.  Returns a dictionary containing
> +    # keys for architecture, board, branch, version.
> +
> +    def get_info_strings(self):
> +        info = dict()
> +        if 'arch' in self:
> +            info['architecture'] = self.arch
> +        else:
> +            info['architecture'] = '<unknown arch>'
> +
> +        if 'board' in self:
> +            info['board'] = self.board
> +        else:
> +            info['board'] = '<unknown board>'
> +
> +        if 'source_branch' in self:
> +            info['branch'] = self.source_branch
> +        else:
> +            info['branch'] = '<unknown branch>'
> +
> +        if 'version' in self:
> +            info['version'] = self.version
> +        else:
> +            info['version'] = '<unknown version>'
> +
> +        return info
> +
Looks OK to me, I may simplify or generalize this code with a later patch.
(e.g. by taking a dictionary of field name -> display name as an optional argument,
or having that dictionary predefined somewhere and defaulting to return
all the keys/values that are *not* specific to the Bunsen storage (such as bunsen_commit_id)
 -- needs some thinking on my part, so please commit as-is for now).

It's worth noting that the set of configuration keys will vary by project,
e.g. for SystemTap 'osver' (the Linux distribution) is a more important field.

> diff --git a/scripts-master/summarize.py b/scripts-master/summarize.py
> new file mode 100755
> index 0000000..e46db18
> --- /dev/null
> +++ b/scripts-master/summarize.py
> @@ -0,0 +1,68 @@
> +#!/usr/bin/env python3
> +
> +# Display a DejaGNU-like test summary, given the bunsen commit
> +# of the desired test run. Optionally also takes a comma-separated
> +# list of glob expressions to limit results.
> +
> +info = "summarize.py <bunsen_commit> [tests]"
> +cmdline_args = [
> +    ('commit', None, '<bunsen_commit>',
> +     "commit to fetch results for"),
> +    ('tests', None, '<test_globs>',
> +     "comma-separated list of glob expressions of tests to summarize")
> +]
> +
> +import sys
> +import bunsen
> +from collections import Counter
> +from pathlib import PurePath
> +
> +# A list of test outcomes in output order.
> +outcome_labels = {
> +    'PASS' : 'expected passes',
> +    'FAIL' : 'unexpected failures',
> +    'XPASS' : 'unexpected successes',
> +    'XFAIL' : 'expected failures',
> +    'KPASS' : 'unknown successes',
> +    'KFAIL' : 'known failures',
> +    'UNTESTED' : 'untested testcases',
> +    'UNRESOLVED' : 'unresolved testcases',
> +    'UNSUPPORTED' : 'unsupported tests',
> +    'ERROR' : 'errors',
> +    'WARNING' : 'warnings'
> +}
> +
> +if __name__ == '__main__':
> +    b = bunsen.Bunsen()
> +    opts = b.cmdline_args(sys.argv, info=info, args=cmdline_args,
> +                          required_args=['commit'], 
> optional_args=['tests'])
> +
> +    testrun = b.testrun(opts.commit)
> +    all_tests = testrun.testcases
> +    found_tests = []
> +    if opts.tests is not None:
> +        for glob in opts.tests.split(','):
> +            found_tests.extend([t for t in all_tests if 
> PurePath(t['name']).match(glob)])
> +    else:
> +        found_tests = all_tests
> +
> +    if found_tests:
> +        info = testrun.get_info_strings()
> +
> +        project = b.tags[0] if len(b.tags) == 1 else '<multiple 
> projects>'
> +        print(f'Summary for commit {opts.commit} of {project} version 
> {info["version"]}')
> +        print(f'from branch {info["branch"]} on {info["architecture"]} 
> using {info["board"]}')
> +        if opts.tests is not None:
> +            print(f'limiting results to tests matching: {opts.tests}')
> +        print()
> +
> +        # Collate results for outcomes
> +        c = Counter(t['outcome'] for t in found_tests)
> +
> +        # We could simply loop over the keys of the Counter, but that 
> would not necessarily give
> +        # us the same output order as DejaGNU itself.
> +        for l in outcome_labels:
> +            if c[l] != 0:
> +                print('# of %-26s %d' % (outcome_labels[l], c[l]))
> +    else:
> +        print(f'found no tests matching \"{opts.tests}\"')
LGTM.

  reply	other threads:[~2020-09-25 15:07 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-24 18:54 Keith Seitz
2020-09-25 15:07 ` Serhei Makarov [this message]
2020-09-25 17:02   ` Keith Seitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=23de1a72-0feb-4937-95a3-07d185e44b02@www.fastmail.com \
    --to=me@serhei.io \
    --cc=bunsen@sourceware.org \
    --cc=keiths@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).