From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from vsmx011.vodafonemail.xion.oxcs.net (vsmx011.vodafonemail.xion.oxcs.net [153.92.174.89]) by sourceware.org (Postfix) with ESMTPS id 608E73858D35 for ; Sat, 8 Aug 2020 04:27:08 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 608E73858D35 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=nexgo.de Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=Stromeko@nexgo.de Received: from vsmx003.vodafonemail.xion.oxcs.net (unknown [192.168.75.197]) by mta-5-out.mta.xion.oxcs.net (Postfix) with ESMTP id 39E9759D2DE for ; Sat, 8 Aug 2020 04:27:07 +0000 (UTC) Received: from Otto (unknown [87.185.208.218]) by mta-7-out.mta.xion.oxcs.net (Postfix) with ESMTPA id 12388539AFF for ; Sat, 8 Aug 2020 04:27:04 +0000 (UTC) From: ASSI To: cygwin-apps@cygwin.com Subject: Re: git repositories for cygwin packaging - please test References: <20e2f046-af24-14b8-b6c4-263f859042b8@dronecode.org.uk> <181ea3fa-1a50-7db4-0009-47ea9af77cdc@dronecode.org.uk> <0def9976-1575-7b03-995b-e4276bc23292@dronecode.org.uk> <87bljm9rae.fsf@Rainer.invalid> Date: Sat, 08 Aug 2020 06:26:59 +0200 In-Reply-To: (Ken Brown via Cygwin-apps's message of "Fri, 7 Aug 2020 18:05:01 -0400") Message-ID: <874kpd225o.fsf@Otto.invalid> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-VADE-STATUS: LEGIT X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: cygwin-apps@cygwin.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Cygwin package maintainer discussion list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Aug 2020 04:27:10 -0000 Ken Brown via Cygwin-apps writes: >>> - take an inordinate amount of time to run (exceeding the resource limits) >> >> That is a problem that comes with CI I think and we didn't really have >> had to consider so far. I have a few packages that I don't run tests on >> by default because the test suite produces hangs or other unstable >> behaviour, but that is dealt with in the src_test function itself. >> If there are really resource hungry tests they usually need to be >> enabled somewhere and one could skip those if the build runs on CI (how >> to find that out?). > > Here's an example where Jon's suggestion would have been useful: While > building php recently, I noticed that the test suite took forever. I > would have been happy to have a way to tell the CI to skip the tests > and avoid exceeding the resource limits. But I wouldn't want to do > that by modifying src_test, because I would still want to run the > tests locally. I think I didn't express myself clearly enough. If you look in the perl-IO-Tty cygport file you'll find this: src_test() { cd ${B} [ "no" != "${CYGPORT_RUN_UNSTABLE_TESTS-no}" ] && cygtest || echo "Unstable test, set CYGPORT_RUN_UNSTABLE_TESTS to run." } This is clearly an exceptional deficiency (even if presumably caused by some border case in Cygwin) that needs to be handled in the package and not a general thing that should be in cygport. But that's not to say that something like CYGPORT_RUN_EXPENSIVE_TESTS could not exist and arguably it should even be directly provided by cygport itself if it was. Now, "expensive" has multiple dimensions, so I'd expect that variable to have a list of values to indicate which of these is considered relevant (like time, memory, network). There'd need to be a matching set of values / limits provided by the local or CI configuration to enable / disable tests as appropriate. CPU wise the CI seems to have a pretty beefy configuration, I've not run into memory problems yet (but didn't check that specifically). So at the moment the relevant limits would be total runtime and maybe network activity (I don't think I have a package that exercises that). But then again, this could change with each run and certainly will change over time, so hard-coding these limits doesn't seem like a good idea. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf rackAttack: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds