From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 676703858C55; Sat, 7 May 2022 20:25:16 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 676703858C55 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from reform (deer0x07.wildebeest.org [172.31.17.137]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id EC39C3000340; Sat, 7 May 2022 22:25:13 +0200 (CEST) Received: by reform (Postfix, from userid 1000) id 0FC1A2E80275; Sat, 7 May 2022 22:25:13 +0200 (CEST) Date: Sat, 7 May 2022 22:25:12 +0200 From: Mark Wielaard To: Jan Beulich Cc: Overseers mailing list , Thomas Fitzsimmons , Dan =?iso-8859-1?Q?Hor=E1k?= , "Frank Ch. Eigler" , binutils@sourceware.org Subject: Re: binutils builder status (Was: Adding binutils to the GNU Toolchain buildbot on sourceware) Message-ID: References: <5c1f217a-109c-2973-6c69-abf412133dee@arm.com> <524b04b7-a78c-7aae-4605-b40f61e6830c@arm.com> <16fe426d-c436-f030-dc43-0e81e7f0e853@arm.com> <20220428141957.GB23335@gnu.wildebeest.org> <20220428162803.GD23335@gnu.wildebeest.org> <20220501163734.GA30898@gnu.wildebeest.org> <88103f52-ea06-de64-5a39-733cf3f62c89@suse.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="zmq8gJucNgWvnoJN" Content-Disposition: inline In-Reply-To: <88103f52-ea06-de64-5a39-733cf3f62c89@suse.com> X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: binutils@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Binutils mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 May 2022 20:25:19 -0000 --zmq8gJucNgWvnoJN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi Jan, On Thu, May 05, 2022 at 09:53:25AM +0200, Jan Beulich wrote: > On 01.05.2022 18:37, Mark Wielaard wrote: > > The gas and binutils testsuites seem clean on all builders. But the ld > > testsuite does see some unexpected failures or passes on some > > builders. It would be great if we could fix these. If not it might > > make sense to run the ld testsuite separately. > > > > Note that you can see the used linux kernel, gcc, binutils versions on > > the workers page: > > https://builder.sourceware.org/buildbot/#/workers > > > > binutils-fedora-s390x > > https://builder.sourceware.org/buildbot/#/builders/binutils-fedora-s390x > > has 2 unexpected ld failures: > > FAIL: Run pr19719 fun undefined > > FAIL: pr26580-3 > > > > binutils-debian-amd64 > > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-amd64 > > has 2 unexpected ld failures: > > FAIL: Run p_align-1b with PIE > > FAIL: Run p_align-1d with -Wl,-z,max-page-size=0x1000 with PIE > > > > binutils-debian-arm64 > > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-arm64 > > has 1 unexpected success: > > XPASS: Run pr19719 fun undefined > > > > binutils-debian-armhf > > https://builder.sourceware.org/buildbot/#/builders/binutils-debian-armhf > > has 8 unexpected ld failures and 7 unexpected successes > > XPASS: Run pr19719 fun undefined > > FAIL: Common symbol override ifunc test 1a > > FAIL: Common symbol override ifunc test 1b > > FAIL: Run pr18841 with libpr18841b.so > > FAIL: Run pr18841 with libpr18841c.so > > FAIL: Run pr18841 with libpr18841bn.so (-z now) > > FAIL: Run pr18841 with libpr18841cn.so (-z now) > > FAIL: Run pr23169a > > FAIL: Run pr23169d > > XPASS: visibility (hidden_undef) (non PIC) > > XPASS: visibility (hidden_undef) (non PIC, load offset) > > XPASS: visibility (hidden_undef) (PIC main, non PIC so) > > XPASS: visibility (protected_undef) (non PIC) > > XPASS: visibility (protected_undef) (non PIC, load offset) > > XPASS: visibility (protected_undef) (PIC main, non PIC so) > > > > The binutils-debian-armhf builder is also the slowest (takes 15 > > minutes). The rest take a few minutes. They should sent email once a > > new failure occurs (or if one if the currently failing builders starts > > passing). > > > > There is also one build and check everything builder > > https://builder.sourceware.org/buildbot/#/builders/binutils-gdb-fedrawhide-x86_64 > > I haven't looked at the test results yet, but they are all stored in > > the bunsendb.git for later analysis. This builder doesn't sent emails > > on bad builds. It also takes a very long time to run (from 1 to 7 hours). > > I've received the first email from that build, as it looks. May I please > ask that this be Cc-ed to individuals (rather than putting them on To: )? > I view To: as reasonable only if there was breakage which was possible to > associate with a particular commit at quite high a probability (which > would require either bisection of there being merely a single commit on > top of what was tested in the prior run). > > To: otoh quite likely wants to include binutils@ ? I think the email you received was for the "recovered" gdb-fedora-s390x builder. This builder failed from the start since the fix on Friday: https://builder.sourceware.org/buildbot/#builders/gdb-fedora-s390x The buildbot assumed everybody who had done a commit since the first fail should hear the happy news that the build now passes. Although it is good news, this was probably too many people. I changed the reporters so only people who committed a change that caused a build to go from success to failure will get a problem report email. Then I added a new reporter that just reports to the mailinglist for any "changed" build (from success to failure or from failure to success). That way you should only be on the To: when you most likely caused the problem. There is one small issue with binutils and gdb because they share a source repository. You might be included in a report about gdb or binutils even if you meant to only change one or the other project. The buildbot knows which directories are part of which project, but some overlap and it doesn't know that a change it didn't explicitly tests when the build for one project fails couldn't have caused that failure. You can write more complicated reporters in python if you want: https://docs.buildbot.net/current/manual/configuration/report_generators/ https://docs.buildbot.net/current/manual/configuration/reporters/mail_notifier.html And as you can see from the attached patch I am probably not a great python programmer. If someones wants to try to clean the master.cfg file up a little to remove all the replications, please do! Cheers, Mark --zmq8gJucNgWvnoJN Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0001-Change-mail-reports-to-only-sent-problems-to-interes.patch" >From c3de0dd3c59818819b9d09c7544ca2feb370770d Mon Sep 17 00:00:00 2001 From: Mark Wielaard Date: Sat, 7 May 2022 22:02:49 +0200 Subject: [PATCH] Change mail reports to only sent problems to interested users. Report for changes (failure to success and success to failure) go to the mailinglist only. Do this by having two reporters. One sents email only to interested users, the other only the the extra recipients (the mailing list). --- builder/master.cfg | 145 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 118 insertions(+), 27 deletions(-) diff --git a/builder/master.cfg b/builder/master.cfg index c131974..988e584 100644 --- a/builder/master.cfg +++ b/builder/master.cfg @@ -1584,6 +1584,7 @@ c['services'].append(mn) # Sent reports about sourceware buildbot updates to overseers # 2022-05-02 ... don't bother overseers@ anymore about this +# Just sent a report to the interested user (committer). generator_sourceware = reporters.BuildStatusGenerator(tags=["sourceware"]) mn_sourceware = reporters.MailNotifier( fromaddr="builder@sourceware.org", @@ -1591,97 +1592,187 @@ mn_sourceware = reporters.MailNotifier( generators=[generator_sourceware]) c['services'].append(mn_sourceware) +# Only sent problem reports to overseers +generator_sourceware_problem = reporters.BuildStatusGenerator( + mode=('problem',), tags=["sourceware"]) +mn_sourceware_problem = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['overseers@sourceware.org'], + generators=[generator_sourceware_problem]) +c['services'].append(mn_sourceware_problem) -# Report for the whole bzip2 tagged builder set +# Problem report for the whole bzip2 tagged builder set generator_bzip2 = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['bzip2']) + mode=('problem',), tags=['bzip2']) mn_bzip2 = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['bzip2-devel@sourceware.org'], generators=[generator_bzip2]) c['services'].append(mn_bzip2) -# Report for the whole debugedit tagged builder set +# Change report for the whole bzip2 tagged builder set +generator_bzip2_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['bzip2']) +mn_bzip2_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['bzip2-devel@sourceware.org'], + generators=[generator_bzip2_change]) +c['services'].append(mn_bzip2_change) + +# Problem report for the whole debugedit tagged builder set generator_debugedit = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['debugedit']) + mode=('problem',), tags=['debugedit']) mn_debugedit = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['debugedit@sourceware.org'], generators=[generator_debugedit]) c['services'].append(mn_debugedit) -# Report for the whole dwz tagged builder set +# Change report for the whole debugedit tagged builder set +generator_debugedit_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['debugedit']) +mn_debugedit_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['debugedit@sourceware.org'], + generators=[generator_debugedit_change]) +c['services'].append(mn_debugedit_change) + +# Problem report for the whole dwz tagged builder set generator_dwz = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['dwz']) + mode=('problem',), tags=['dwz']) mn_dwz = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['dwz@sourceware.org'], generators=[generator_dwz]) c['services'].append(mn_dwz) -# Report for the whole elfutils tagged builder set +# Change report for the whole dwz tagged builder set +generator_dwz_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['dwz']) +mn_dwz_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['dwz@sourceware.org'], + generators=[generator_dwz_change]) +c['services'].append(mn_dwz_change) + +# Problem report for the whole elfutils tagged builder set generator_elfutils = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['elfutils']) + mode=('problem',), tags=['elfutils']) mn_elfutils = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['elfutils-devel@sourceware.org'], generators=[generator_elfutils]) c['services'].append(mn_elfutils) -# Report for the whole gccrust tagged builder set +# Change report for the whole elfutils tagged builder set +generator_elfutils_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['elfutils']) +mn_elfutils_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['elfutils-devel@sourceware.org'], + generators=[generator_elfutils_change]) +c['services'].append(mn_elfutils_change) + +# Problem report for the whole gccrust tagged builder set generator_gccrust = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['gccrust']) + mode=('problem',), tags=['gccrust']) mn_gccrust = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['gcc-rust@gcc.gnu.org'], generators=[generator_gccrust]) c['services'].append(mn_gccrust) -# Report for the whole binutils tagged builder set +# Change report for the whole gccrust tagged builder set +generator_gccrust_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['gccrust']) +mn_gccrust_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['gcc-rust@gcc.gnu.org'], + generators=[generator_gccrust_change]) +c['services'].append(mn_gccrust_change) + +# Problem report for the whole binutils tagged builder set generator_binutils = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['binutils']) + mode=('problem',), tags=['binutils']) mn_binutils = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['binutils@sourceware.org'], generators=[generator_binutils]) c['services'].append(mn_binutils) -# Report for the whole gdb tagged builder set +# Change report for the whole binutils tagged builder set +generator_binutils_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['binutils']) +mn_binutils_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['binutils@sourceware.org'], + generators=[generator_binutils_change]) +c['services'].append(mn_binutils_change) + +# Problem report for the whole gdb tagged builder set generator_gdb = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['gdb']) + mode=('problem',), tags=['gdb']) mn_gdb = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['gdb-testers@sourceware.org'], generators=[generator_gdb]) c['services'].append(mn_gdb) -# Report for the whole libabigail tagged builder set +# Change report for the whole gdb tagged builder set +generator_gdb_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['gdb']) +mn_gdb_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['gdb-testers@sourceware.org'], + generators=[generator_gdb_change]) +c['services'].append(mn_gdb_change) + +# Problem report for the whole libabigail tagged builder set generator_libabigail = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['libabigail']) + mode=('problem',), tags=['libabigail']) mn_libabigail = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['libabigail@sourceware.org'], generators=[generator_libabigail]) c['services'].append(mn_libabigail) -# Report for the whole valgrind tagged builder set +# Changereport for the whole libabigail tagged builder set +generator_libabigail_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['libabigail']) +mn_libabigail_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['libabigail@sourceware.org'], + generators=[generator_libabigail_change]) +c['services'].append(mn_libabigail_change) + +# Problem report for the whole valgrind tagged builder set generator_valgrind = reporters.BuildSetStatusGenerator( - mode=('change',), tags=['valgrind']) + mode=('problem',), tags=['valgrind']) mn_valgrind = reporters.MailNotifier( fromaddr="builder@sourceware.org", sendToInterestedUsers=True, - extraRecipients=['valgrind-testresults@lists.sourceforge.net'], generators=[generator_valgrind]) c['services'].append(mn_valgrind) +# Change report for the whole valgrind tagged builder set +generator_valgrind_change = reporters.BuildSetStatusGenerator( + mode=('change',), tags=['valgrind']) +mn_valgrind_change = reporters.MailNotifier( + fromaddr="builder@sourceware.org", + sendToInterestedUsers=False, + extraRecipients=['valgrind-testresults@lists.sourceforge.net'], + generators=[generator_valgrind_change]) +c['services'].append(mn_valgrind_change) + ####### PROJECT IDENTITY -- 2.30.2 --zmq8gJucNgWvnoJN--