From: Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
To: gcc-patches@gcc.gnu.org
Cc: Diego Novillo <dnovillo@google.com>, Doug Evans <dje@google.com>,
Maxim Kuvyrkov <maxim.kuvyrkov@linaro.org>
Subject: [PATCH 03/12] [contrib] validate_failures.py: Read in manifest when comparing build dirs
Date: Fri, 2 Jun 2023 15:20:43 +0000 [thread overview]
Message-ID: <20230602152052.1874860-4-maxim.kuvyrkov@linaro.org> (raw)
In-Reply-To: <20230602152052.1874860-1-maxim.kuvyrkov@linaro.org>
This allows comparison of two build directories with a manifest
listing known flaky tests on the side.
---
contrib/testsuite-management/validate_failures.py | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/contrib/testsuite-management/validate_failures.py b/contrib/testsuite-management/validate_failures.py
index 7351ba120b7..4733dd89dcb 100755
--- a/contrib/testsuite-management/validate_failures.py
+++ b/contrib/testsuite-management/validate_failures.py
@@ -420,9 +420,10 @@ def CollectSumFiles(builddir):
return sum_files
-def GetResults(sum_files):
+def GetResults(sum_files, build_results = None):
"""Collect all the test results from the given .sum files."""
- build_results = ResultSet()
+ if build_results == None:
+ build_results = ResultSet()
for sum_fname in sum_files:
print('\t%s' % sum_fname)
build_results |= ParseSummary(sum_fname)
@@ -567,8 +568,15 @@ def CompareBuilds():
sum_files = GetSumFiles(_OPTIONS.results, _OPTIONS.build_dir)
actual = GetResults(sum_files)
+ clean = ResultSet()
+
+ if _OPTIONS.manifest:
+ manifest_path = GetManifestPath(srcdir, target, True)
+ print('Manifest: %s' % manifest_path)
+ clean = GetManifest(manifest_path)
+
clean_sum_files = GetSumFiles(_OPTIONS.results, _OPTIONS.clean_build)
- clean = GetResults(clean_sum_files)
+ clean = GetResults(clean_sum_files, clean)
return PerformComparison(clean, actual, _OPTIONS.ignore_missing_failures)
--
2.34.1
next prev parent reply other threads:[~2023-06-02 15:21 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-02 15:20 [contrib] Extend and improve validate_failures.py Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 01/12] [contrib] validate_failures.py: Avoid testsuite aliasing Maxim Kuvyrkov
2023-06-03 15:17 ` Jeff Law
2023-06-05 14:06 ` Maxim Kuvyrkov
2023-09-26 15:46 ` Bernhard Reutner-Fischer
2023-09-27 14:47 ` Maxim Kuvyrkov
2023-10-03 10:37 ` rep.dot.nop
2023-11-02 13:31 ` Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 02/12] [contrib] validate_failures.py: Support expiry attributes in manifests Maxim Kuvyrkov
2023-06-02 15:20 ` Maxim Kuvyrkov [this message]
2023-06-02 15:20 ` [PATCH 04/12] [contrib] validate_failures.py: Simplify GetManifestPath() Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 05/12] [contrib] validate_failures.py: Add more verbosity levels Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 06/12] [contrib] validate_failures.py: Be more stringent in parsing result lines Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 07/12] [contrib] validate_failures.py: Use exit code "2" to indicate regression Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 08/12] [contrib] validate_failures.py: Support "$tool:" prefix in exp names Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 09/12] [contrib] validate_failures.py: Improve error output Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 10/12] [contrib] validate_failures.py: Add new option --invert_match Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 11/12] [contrib] validate_failures.py: Add "--expiry_date YYYYMMDD" option Maxim Kuvyrkov
2023-06-02 15:20 ` [PATCH 12/12] [contrib] validate_failures.py: Ignore stray filesystem paths in results Maxim Kuvyrkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230602152052.1874860-4-maxim.kuvyrkov@linaro.org \
--to=maxim.kuvyrkov@linaro.org \
--cc=dje@google.com \
--cc=dnovillo@google.com \
--cc=gcc-patches@gcc.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).