On 01/12/20 10:45 +0100, Christophe Lyon wrote: >On Mon, 30 Nov 2020 at 15:58, Jonathan Wakely wrote: >> >> On 27/11/20 21:17 +0100, Christophe Lyon via Libstdc++ wrote: >> >On Fri, 27 Nov 2020 at 17:13, Jonathan Wakely via Gcc-patches >> > wrote: >> >> >> >> The default for the GCC testsuite is 300, i.e. 5 minutes, which is the >> >> same as the DejaGnu default. >> >> >> >> Libstdc++ overrides this to 600, i.e. 10 minutes. >> >> >> >> This seems ridiculously long. If any test takes that long on modern >> >> hardware, something is wrong. We've seen this a lot recently with >> >> buggy tests, and waiting for them to FAIL is tedious. >> >> >> >> I've already made libstdc++.exp respect the user's setting in >> >> ~/.dejagnurc or the global site.exp file. This means anybody testing >> >> on slow simulators or old hardware can choose their own timeout. >> >> >> >> I've added dg-timeout-factor to the slowest std::regex tests and have >> >> a patch to do it for the PSTL tests, which both take far too long to >> >> compile. That means you can choose a sensible timeout appropriate for >> >> most tests (e.g. 60 seconds) and not get spurious failures from the >> >> few dozen tests which are just very slow. >> >> >> >> I'd like to change the default to 6 minutes. If that goes well, I'd >> >> like to lower it even further. >> >> >> >> The main benefit of this will be that buggy tests which hang will get >> >> killed sooner, so we waste less time waiting for the inevitable >> >> timeout. >> >> >> > >> >I think that's a good idea, I did have problems sometimes when >> >many tests timed out, causing the whole 'make check' to be >> >killed before completion by our compute farm management system. >> >> Thanks for the feedback. I've pushed this patch now. >> >> It's been tested on powercp64le-linux, x86_64-linux, aarch64-linux, >> sparc-solaris and powerpc-aix. They were all fine with much lower >> defaults (e.g. 120 seconds). Let's see how this goes for people >> testing on older or less powerful hardware. >> > >FTR, I've seen two occurrences of a random timeout: >WARNING: program timed out. >27_io/basic_istream/get/wchar_t/lwg3464.cc execution test (reason: NONE) >FAIL: 27_io/basic_istream/get/wchar_t/lwg3464.cc execution test > >when testing for aarch64-none-elf with -mabi=ilp32 using Arm's >Foundation Model as simulator (an old release). Yes, that test only runs for target { ! lp64 } and does quite a lot of work. It should compile quite quickly, but takes a long time to run compared to most tests. I've add dg-timeout-factor to it and its narrow char counterpart. Tested x86_64-linux, pushed to trunk.