Re: Buildbot failure in Wildebeest Builder on whole buildset

[Mark Wielaard <mark@klomp.org>]

Hi Marc,

On Thu, Feb 17, 2022 at 08:46:01PM +0100, Marc via Gcc-rust wrote:
> buildbot@builder.wildebeest.org writes:
> 
> > Build Reason: <unknown>
> > Blamelist: Philip Herron <philip.herron@embecosm.com>
> 
> g++  -fno-PIE -c  -DIN_GCC_FRONTEND -g -O2     -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -I. -Irust -I../../gccrs/gcc -I../../gccrs/gcc/rust -I../../gccrs/gcc/../include -I../../gccrs/gcc/../libcpp/include -I../../gccrs/gcc/../libcody  -I../../gccrs/gcc/../libdecnumber -I../../gccrs/gcc/../libdecnumber/dpd -I../libdecnumber -I../../gccrs/gcc/../libbacktrace   -o rust/rust-hir-full-test.o -MT rust/rust-hir-full-test.o -MMD -MP -MF rust/.deps/rust-hir-full-test.TPo -std=c++11 -Wno-unused-parameter -Werror=overloaded-virtual -I ../../gccrs/gcc/rust -I ../../gccrs/gcc/rust/lex -I ../../gccrs/gcc/rust/parse -I ../../gccrs/gcc/rust/ast -I ../../gccrs/gcc/rust/analysis -I ../../gccrs/gcc/rust/backend -I ../../gccrs/gcc/rust/expand -I ../../gccrs/gcc/rust/hir/tree -I ../
>  ../gccrs/gcc/rust/hir -I ../../gccrs/gcc/rust/resolve -I ../../gccrs/gcc/rust/util -I ../../gccrs/gcc/rust/typecheck -I ../../gccrs/gcc/rust/lint ../../gccrs/gcc/rust/hir/tree/rust-hir-full-test.cc
> command timed out: 1200 seconds without output running [b'make', b'-j4'], attempting to kill
> process killed by signal 9
> program finished with exit code -1
> elapsedTime=1425.203557
> 
> Looks suspicious. Is it possible that the VM got stuck ?

It did indeed. The machine only has 4GB of memory and 2GB swap. With
make -j4 there are 4 processes each taking ~1GB and it is just
swapping:

  926 root      20   0       0      0      0 R  97.6   0.0 135:41.18 kswapd0
15794 buildbot  20   0 1539436 928056      0 D   2.4  24.5   1:41.92 cc1plus
15817 buildbot  20   0  781108 674280    328 D   2.4  17.8   1:08.31 cc1plus
15792 buildbot  20   0 1250264 763912    384 D   2.1  20.2   1:40.62 cc1plus
15800 buildbot  20   0 1427104 784820    132 D   2.1  20.7   1:41.67 cc1plus

I'll reconfigure with make -j3 to see if that helps.

I don't know why we are suddenly seeing this, did the code get that
much bigger recently?

Cheers,

Mark