From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bunsen-return-4-listarch-bunsen=sourceware.org@sourceware.org>
Received: (qmail 20116 invoked by alias); 9 Oct 2019 17:00:22 -0000
Mailing-List: contact bunsen-help@sourceware.org; run by ezmlm
Precedence: bulk
List-Post: <mailto:bunsen@sourceware.org>
List-Help: <mailto:bunsen-help@sourceware.org>
List-Subscribe: <mailto:bunsen-subscribe@sourceware.org>
List-Id: <bunsen.sourceware.org>
Sender: bunsen-owner@sourceware.org
Received: (qmail 6149 invoked by uid 48); 9 Oct 2019 17:00:04 -0000
From: "me at serhei dot io" <sourceware-bugzilla@sourceware.org>
To: bunsen@sourceware.org
Subject: [Bug bunsen/25091] New: [v1.0] analysis: which regressions are truly
 new?
Date: Tue, 01 Jan 2019 00:00:00 -0000
X-Bugzilla-Reason: AssignedTo
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: bunsen
X-Bugzilla-Component: bunsen
X-Bugzilla-Version: unspecified
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: normal
X-Bugzilla-Who: me at serhei dot io
X-Bugzilla-Status: NEW
X-Bugzilla-Resolution: 
X-Bugzilla-Priority: P2
X-Bugzilla-Assigned-To: bunsen at sourceware dot org
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Flags: 
X-Bugzilla-Changed-Fields: bug_id short_desc product version bug_status
 bug_severity priority component assigned_to reporter target_milestone
Message-ID: <bug-25091-12167@http.sourceware.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: http://sourceware.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-SW-Source: 2019-q4/txt/msg00003.txt.bz2

https://sourceware.org/bugzilla/show_bug.cgi?id=3D25091

            Bug ID: 25091
           Summary: [v1.0] analysis: which regressions are truly new?
           Product: bunsen
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: bunsen
          Assignee: bunsen at sourceware dot org
          Reporter: me at serhei dot io
  Target Milestone: ---

Due to flaky tests flipping back and forth, the regression info generated by
+diff_commits is very noisy. In my presentation I described the next step as
'identifying and filtering out flaky testcases', but a more useful and simp=
le
approach might be to search the entire history (by regression) and then
identify when a new regression *first* appears (is 'truly new').

This could be done either for the entire history, or for an n-month or n-co=
mmit
sliding window over the history (i.e. show a regression if it hasn't appear=
ed
during previous n months/commits).

--=20
You are receiving this mail because:
You are the assignee for the bug.