public inbox for bunsen@sourceware.org
 help / color / mirror / Atom feed
* bunsen (re)design discussion #0: list of topics, review of terminology
@ 2022-03-07 18:59 Serhei Makarov
  2022-03-07 19:03 ` Serhei Makarov
  0 siblings, 1 reply; 7+ messages in thread
From: Serhei Makarov @ 2022-03-07 18:59 UTC (permalink / raw)
  To: Bunsen

I'm in the process of re-evaluating and re-working some core aspects of the Bunsen code, with input from Frank Ch. Eigler. This will result in several public emails like this one as I sort through various concerns.

Initial list of topics I need to cover is below. Incidentally, it approximately matches the sequence in which Bunsen concepts would need to be introduced in a Bunsen man page or tutorial.

(0) Review of Terminology
(1) Testrun Representation
  - version identifiers (source_commit, package_nvr) & sorting
  - configuration & whether to allow 'arbitrary' metadata
(2) Repository Layout
  - splitting testruns across branches, fche's suggestion
  - speculation: should we adopt SQLite? or borrow ideas from Django ORM?
(3) Analysis Pipeline
  - splitting querying from analysis from formatting
  - Index class represents the result of a query
  - Table class represents data for formatting (as plaintext, HTML, etc.)
  - configuration file format: borrowing from the 'Makefile' metaphor to describe items of data and reports Bunsen produces, and how to refresh each item when its dependencies change

===

(0) Review of Terminology

Bunsen repo: collection of test result data managed by Bunsen

source repo: Git repo with the source code of the project being tested

project: identifier of the project being tested, e.g. 'gdb', 'systemtap'
- One Bunsen repo can include data from multiple projects.
   - By default, Bunsen analysis is run on data from one project at a time. This can be overridden by providing a list of projects e.g. "--project=systemtap,systemtap-incomplete".
   - This is useful for a setup such as gcc-jenkins http://gcc.gnu.org/jenkins / http://3.14.90.209:8080/ which produces results for several projects such as gcc,ld,... in a single testsuite run.
   - This is also useful when we want to split out a category of data for separate analysis (e.g. projectX-incomplete for projectX testsuite runs that crashed partway through and therefore don't include full results for all testcases).

testlog: test result logs (a file tree of plaintext files produced by a testsuite run)
- For example, testlogs from a SystemTap testsuite run might include systemtap.{sum,log} from the DejaGNU testsuite, plus additional diagnostics such as systemtap.dmesg, stap-report, sysinfo.

testrun: the parsed representation of a testsuite run, currently stored in JSON format
- More details on what this includes in the next email.

(May have further clarifications / definitions of terms in replies to this thread.)

All the best,
    Serhei
    http://serhei.io

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2022-04-09  2:26 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-07 18:59 bunsen (re)design discussion #0: list of topics, review of terminology Serhei Makarov
2022-03-07 19:03 ` Serhei Makarov
2022-04-07 16:57   ` Keith Seitz
2022-04-08 23:13     ` Serhei Makarov
2022-04-09  1:56       ` Frank Ch. Eigler
2022-04-09  2:22         ` Serhei Makarov
2022-04-09  2:26           ` Frank Ch. Eigler

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).