From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gnu.wildebeest.org (gnu.wildebeest.org [45.83.234.184]) by sourceware.org (Postfix) with ESMTPS id 900003858D20 for ; Thu, 17 Feb 2022 22:03:20 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 900003858D20 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=klomp.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=klomp.org Received: from reform (deer0x16.wildebeest.org [172.31.17.152]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gnu.wildebeest.org (Postfix) with ESMTPSA id A4F043039B04; Thu, 17 Feb 2022 23:03:18 +0100 (CET) Received: by reform (Postfix, from userid 1000) id BF1FD2E808DA; Thu, 17 Feb 2022 23:03:17 +0100 (CET) Date: Thu, 17 Feb 2022 23:03:17 +0100 From: Mark Wielaard To: Marc Cc: gcc-rust@gcc.gnu.org Subject: Re: Buildbot failure in Wildebeest Builder on whole buildset Message-ID: References: <20220217192618.72371801D13@builder.wildebeest.org> <87wnht1dl2.fsf@arrakis.kataplop.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc-rust@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: gcc-rust mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Feb 2022 22:03:22 -0000 Hi, On Thu, Feb 17, 2022 at 10:05:29PM +0100, Mark Wielaard wrote: > On Thu, Feb 17, 2022 at 08:46:01PM +0100, Marc via Gcc-rust wrote: > > ../gccrs/gcc/rust/hir -I ../../gccrs/gcc/rust/resolve -I ../../gccrs/gcc/rust/util -I ../../gccrs/gcc/rust/typecheck -I ../../gccrs/gcc/rust/lint ../../gccrs/gcc/rust/hir/tree/rust-hir-full-test.cc > > command timed out: 1200 seconds without output running [b'make', b'-j4'], attempting to kill > > process killed by signal 9 > > program finished with exit code -1 > > elapsedTime=1425.203557 > > > > Looks suspicious. Is it possible that the VM got stuck ? > > It did indeed. The machine only has 4GB of memory and 2GB swap. With > make -j4 there are 4 processes each taking ~1GB and it is just > swapping: > > 926 root 20 0 0 0 0 R 97.6 0.0 135:41.18 kswapd0 > 15794 buildbot 20 0 1539436 928056 0 D 2.4 24.5 1:41.92 cc1plus > 15817 buildbot 20 0 781108 674280 328 D 2.4 17.8 1:08.31 cc1plus > 15792 buildbot 20 0 1250264 763912 384 D 2.1 20.2 1:40.62 cc1plus > 15800 buildbot 20 0 1427104 784820 132 D 2.1 20.7 1:41.67 cc1plus > > I'll reconfigure with make -j3 to see if that helps. It didn't. With -j3 the oom-killer kicks in (dunno why not with -j4): [1643873.684998] Out of memory: Killed process 16625 (cc1plus) total-vm:1764740kB, anon-rss:1040744kB, file-rss:0kB, shmem-rss:0kB, UID:998 pgtables:3464kB oom_score_adj:0 [1643873.992761] oom_reaper: reaped process 16625 (cc1plus), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB So now using make -j2 which does succeed. https://builder.wildebeest.org/buildbot/#/builders/58/builds/1655 Very odd. But apparently for some reason 4GB isn't enough anymore to use -j3 or -j4. Which is unfortunate because this board doesn't take more than 4GB memory. Cheers, Mark