Re: Initial findings of bunsen performance and questions

[Keith Seitz <keiths@redhat.com>]

On 9/16/20 4:09 PM, Serhei Makarov wrote:

> On Wed, Sep 16, 2020, at 6:18 PM, Keith Seitz via Bunsen wrote:
>> The first question to ask is: Is this an oversimplified/naive implementation
>> of this script?
> It looks right to me. I briefly suspected the lambda might do some additional
> object copying, but that doesn't seem to be the case.

I profiled the code at one time to ascertain where all the time was being
spent. I don't have the specific results handy, but IIRC, most of the time
was being spent in Testrun.__init__, specifically:

            for field in defer_fields:
                self[field] = self._deserialize_testrun_field(field, self[field],
                                                              cursor_commit_ids)


> I suspect my code for building the repo has some bug when using consolidate_pass=False.
> Could you place the Git/JSON repo you built somewhere I have access to?

Yes, I will send you (privately) what I've been benchmarking. This is nearly unaltered
bunsen repo, but I've added a few simple patches to fix some existing problems.

> (Also, you could try the +diff_runs script on your repo. If the JSON
> parsing is the source of the slowdown and reading one run took 20s, 
> logically reading two runs would take you 40s.)

That is exactly what happens:

$ time ./bunsen.py +diff_runs 42563db d2a72bd
...
real    0m57.300s
user    0m49.599s
sys     0m17.683s

> IMO the comparison has to be done with 100s to 1000s of similar test runs
> since Git's de-duplication must be compared to whatever SQLite does,
> at that scale of data.
> I doubt it's important though, for this use case we have disk space to burn
> and the query speedup even justifies keeping both forms of storage.

Indeed -- the sample space is too small, but I just wanted to get a feeling of
how this is going.

>> Is this an approach that seems viable as a supplement (replacement?)
>> to JSON output? Is this approach something worth pursuing?
> Definitely worth pursuing due to the aforementioned possibility of 'column queries'
> which I don't see any way of handling well with the design I currently have.
> 
> I'm not sure if SQLite is better used as a replacement for the JSON/Git storage
> or as a supplemental cache built from it and used to speed up queries.
> (Also, the original log files must be retained in any case.)

All my current proof-of-concept does is replace the *test* data .JSON -> .db.
So there are still JSON files that are used to describe the Testrun metadata, e.g.
I haven't attempted to change that since I haven't fully investigated it.

Right now, my plan is to make this an optional/configurable option. It really
only needs to be configured when data is imported. We should be able to otherwise
handle this transparently behind the scenes.

So I will just continue on my way and try to get something review-ready.

Thank you for your input!

Keith