Re: bunsen to assist ci/cd testsuite historical analysis needs

[Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>]

> On Jan 11, 2024, at 21:51, Frank Ch. Eigler <fche@redhat.com> wrote:
> 
> Hi -
> 
>>> OK, will check that out.  Is the git repo with results also visible somewhere?
>> [...]
>> [2] https://git.linaro.org/toolchain/ci/base-artifacts/tcwg_gcc_check/master-aarch64.git/
> 
> Just for reference, after a quick glance at this, it's close to but
> not quite verbatim usable by bunsen:
> 
> - the .xz compression/extension would interfere with extension based
>  matching

We can fix that.

> 
> - separating the .log and .sum files into different directories stops
>  bunsen's crossmatching of the two; bunsen can accept the build tree
>  raw locations of these files, without the "sumfiles" / "00-sumfiles"
>  type of reorg

We partially re-run the testsuites when results change unexpectedly.  This allows us to detect and ignore flaky tests.  00-sumfiles/ is an auxiliary directory that contains all run and re-run logs and .sum files; sumfiles/ directory contains the final [synthetic] .sum files that are a merge of all re-run .sum files.  All this is to say that we don't necessarily have the matching logs for the final [synthetic] .sum files.

What information do you extract from the .log files?

> 
> - for buildbot builds on sourceware, we collect & parse the autoconf
>  config.log files as first class content, rather than just informal
>  console logs

We can gather config.log files.

> 
> - bunsen would love to receive metadata about the build in little
>  .bunsen.FOO text files, e.g. see:
>  https://sourceware.org/git/?p=bunsendb.git;a=commit;h=613dd3b381b0910ebd5036fcf42761e2cead12ee
> 

Ack.

> 
> I assume you had good reasons for .xz compressing these files inside
> the git repo ... but for what it's worth, we find that letting normal
> git-repack transparently manage all the compression of plain text
> payloads gives us excellent results.
> 

These are also uploaded outside of git repos, but it's not a problem to push uncompressed files into git.

Kind regards,

--
Maxim Kuvyrkov
https://www.linaro.org