From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id D69EF385627A for ; Fri, 10 Jun 2022 15:59:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org D69EF385627A Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654876774; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=DPQXT5fIvfOVY8kmEmMKaN+QezbKpp0+dKmWJaT0sYw=; b=gvyoLQ/uQNTlgQCh7XHqLtwnqPZzL2KU2VFCXrqS52aV7Vkn6HxSlCVlbEJHUH8PiMP0bn WQq+IK2zZjKYy+IAgiXgaZ9TVW9Uieg5Jy37yYsbcXXMNT/PDn8B8Wfo/U3BUt4MG1zTdH 8ZgpSqc6wSwIJFUYwERh4AAO6I49jx8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-Wd-wky7aM5Sy3CXsAfFeLg-1; Fri, 10 Jun 2022 11:59:31 -0400 X-MC-Unique: Wd-wky7aM5Sy3CXsAfFeLg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 43B1C1C05ABA; Fri, 10 Jun 2022 15:59:31 +0000 (UTC) Received: from redhat.com (unknown [10.2.16.9]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3325E40B40C8; Fri, 10 Jun 2022 15:59:31 +0000 (UTC) Received: from fche by redhat.com with local (Exim 4.94.2) (envelope-from ) id 1nzh2o-0007ns-Bd; Fri, 10 Jun 2022 11:59:30 -0400 Date: Fri, 10 Jun 2022 11:59:30 -0400 From: "Frank Ch. Eigler" To: bunsen@sourceware.org, buildbot@sourceware.org Subject: announcing prototype buildbot<->bunsendb<->bunsenweb integration Message-ID: <20220610155930.GC10842@redhat.com> MIME-Version: 1.0 User-Agent: Mutt/1.12.0 (2019-05-25) X-Scanned-By: MIMEDefang 2.84 on 10.11.54.1 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=fche@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Status: No, score=-10.1 required=5.0 tests=BAYES_00, DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, KAM_SHORT, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_NONE, TXREP, T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: buildbot@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "The https://builder.sourceware.org/ buildbot" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Jun 2022 15:59:36 -0000 Hi - Not long after mjw's buildbot setup in April here on sourceware, we've coupled to it a testsuite log file collection system. This has now reached a prototyping stage where it is more interactively browsable and might become useful for actual test problem diagnosis. Here's how that stuff works: 0) sourceware has a buildbot master & a fleet of donated workers, as mjw announced over the last month or two, and here: https://gnu.wildebeest.org/~mark/sourceware.txt 1) most buildbot job steps include a bunsen_logfile_upload_steps() operation, which collects named log/sum/trs type files from the workers and pushes them into git://sourceware.org/git/bunsendb.git under a tag name that corresponds to the build https://sourceware.org/git/?p=bunsendb.git;a=blob;f=README 2) a bunsendb.timer/.service job pair on sourceware regularly polls this git repository, and runs a bunsen processing pipeline to analyze all the logs and deposit results into an sqlite database file back on sourceware https://sourceware.org/git/?p=bunsen.git;a=blob;f=README;hb=refs/heads/fche/bunsenql /sourceware/projects/bunsen-home/bunsenweb/bunsenql.sqlite3 (on sourceware) /sourceware/projects/bunsen-home/bunsenql (bunsenql branch code checkout) 3) a bunsenweb.service job on sourceware activates the r-httpd-browse part of bunsen to let people use a browser to query / browse both the analysis database and the underlying log files. This part is quite early, light on eye candy, will change rapidly. But it may already be useful, so we're decloaking: https://builder.sourceware.org/testruns/ << entry point Some things to try. All this is computed on the fly from the database, but most operations are surprisingly quick. - click on a testrun (row), observe the metadata associated with that testrun and the list of files, each of which may be viewed - experiment with the filter fields at the top to require presence of dejagnu .exp file names (ignore the automake trs stuff for now), or metadata; each uses sql LIKE pattern syntax for the operations https://www.sqlite.org/lang_expr.html#like e.g. https://tinyurl.com/43ntbsw2 to filter to some most recent binutils-gdb gcc/clang rawhide full testsruns - click checkbox on two or more testruns to compare, hit "diff", getting a difference report between the runs e.g. https://tinyurl.com/2p88vu2h to compare some recent gdb builds - from there, observe the comparison grid; in each cell, links are generally given to the particular segment of the log/sum files where the test case failed/passed differently from its peers e.g. https://tinyurl.com/2p8evcca Lots of things will be coming here, including taking advantage of testrun cluster analysis that's already being done, a per-testrun testcase search/browse engine, other search operators, testsuite summary (vs detail) grids, who knows, ideas welcome! Note that 2) and 3) can be also done independently on your own workstation, if for example you git-clone the sourceware bunsendb. Or all of 1) 2) 3) if you have your own bunch of log files to import into a whole separate installation. The README files provide some instruction, and we'll have more man pages before long. - FChE