From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 3F3763858C5F for ; Wed, 6 Mar 2024 22:21:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 3F3763858C5F Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 3F3763858C5F Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=45.83.234.184 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709763691; cv=none; b=MegOEdHpUcj8wOPuM1d5PcqEc+PSyNCGt+95F9wVolI4ZdkrkMPTCYDMEvU5s8iS97m1jmot2X95/oeigeGVMNgD785wNlmYZxcAEVihL3D7YMGJOQw9klxjBMGf2VGAHyD1kBGvh5s6uNfd2aW1nzy7/vT/N0TR0biBensgXBw= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1709763691; c=relaxed/simple; bh=IVqG9B04P39Jwe+NTkfzqq9f/ySQeHrK4YtNwY67Vms=; h=Date:From:To:Subject:Message-ID:MIME-Version; b=kqBj8gzt1YXvM/n188dAE8ecIbPW/9BTtJVScOsfl3avpU67s2l64sMeZnBigKyxx3d5mg12cdDaNJUuKAR797N2sVEAj4/9LMfZALyXJhQsyKA/CGRUMqA72liJINfUgHhBQeUlF9P8j8lCh2iRFqbNh5H6RM6dnMtSY/BACG8= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by gnu.wildebeest.org (Postfix, from userid 1000) id 4BE2F3000472; Wed, 6 Mar 2024 23:21:28 +0100 (CET) Date: Wed, 6 Mar 2024 23:21:28 +0100 From: Mark Wielaard To: "Frank Ch. Eigler" Cc: buildbot@sourceware.org, nickc@redhat.com Subject: Re: patch: use "make -k check" broadly Message-ID: <20240306222128.GE24213@gnu.wildebeest.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-9.0 required=5.0 tests=BAYES_00,GIT_PATCH_0,JMQ_SPF_NEUTRAL,KAM_DMARC_STATUS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Wed, Mar 06, 2024 at 12:14:31PM -0500, Frank Ch. Eigler wrote: > The following diff gets the annobin and a few other package > make_check* steps to use "make -k check" rather than "make check", so > that early partial failures do not doom the entire run. Many other > package builders already use the equivalent. Yes that makes sense. I do hope most testsuites are normally zero fail, so any failure would indicate an issue. But we would like to get as much test results as possible in the CI builders. And make -k check still makes the command fail if any make target fails. It just doesn't bail out early. Thanks, Mark > diff --git a/builder/master.cfg b/builder/master.cfg > index e4cd0802a051..d0b046f7af19 100644 > --- a/builder/master.cfg > +++ b/builder/master.cfg > @@ -1194,26 +1194,26 @@ make_pdf_step = steps.Compile( > command=["make", "pdf"], > name="make pdf") > make_check_step = steps.Test( > - command=addOutputSync.withArgs(["make", "check", > + command=addOutputSync.withArgs(["make", "-k", "check", > util.Interpolate('-j%(prop:ncpus)s')]), > name="make check", haltOnFailure=False, flunkOnFailure=True) > # make check explicitly not using -j > make_single_check_step = steps.Test( > - command=["make", "check"], > + command=["make", "-k", "check"], > name="make check") > make_single_test_suite_step = steps.Test( > - command=["make", "check"], > + command=["make", "-k" "check"], > name="make check", > logfiles={"test-suite.log": "tests/test-suite.log"}) > # Same as make_check_step but with tests/test-suite.log recorded > make_check_test_suite_step = steps.Test( > - command=addOutputSync.withArgs(["make", "check", > + command=addOutputSync.withArgs(["make", "-k", "check", > util.Interpolate('-j%(prop:ncpus)s')]), > name="make check", haltOnFailure=False, flunkOnFailure=True, > logfiles={"test-suite.log": "tests/test-suite.log"}) > # Same but with tests/testsuite.log (note, no dash) > make_check_testsuite_step = steps.Test( > - command=addOutputSync.withArgs(["make", "check", > + command=addOutputSync.withArgs(["make", "-k", "check", > util.Interpolate('-j%(prop:ncpus)s')]), > name="make check", haltOnFailure=False, flunkOnFailure=True, > logfiles={"testsuite.log": "tests/testsuite.log"}) > > > - FChE