Re: [RFC] Decrease default timeout for libstdc++ tests to 6 minutes

[Christophe Lyon <christophe.lyon@linaro.org>]

On Fri, 27 Nov 2020 at 17:13, Jonathan Wakely via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> The default for the GCC testsuite is 300, i.e. 5 minutes, which is the
> same as the DejaGnu default.
>
> Libstdc++ overrides this to 600, i.e. 10 minutes.
>
> This seems ridiculously long. If any test takes that long on modern
> hardware, something is wrong. We've seen this a lot recently with
> buggy tests, and waiting for them to FAIL is tedious.
>
> I've already made libstdc++.exp respect the user's setting in
> ~/.dejagnurc or the global site.exp file. This means anybody testing
> on slow simulators or old hardware can choose their own timeout.
>
> I've added dg-timeout-factor to the slowest std::regex tests and have
> a patch to do it for the PSTL tests, which both take far too long to
> compile. That means you can choose a sensible timeout appropriate for
> most tests (e.g. 60 seconds) and not get spurious failures from the
> few dozen tests which are just very slow.
>
> I'd like to change the default to 6 minutes. If that goes well, I'd
> like to lower it even further.
>
> The main benefit of this will be that buggy tests which hang will get
> killed sooner, so we waste less time waiting for the inevitable
> timeout.
>

I think that's a good idea, I did have problems sometimes when
many tests timed out, causing the whole 'make check' to be
killed before completion by our compute farm management system.

Thanks

Christophe

>
> Here's the proposed patch for completeness:
>
> --- a/libstdc++-v3/testsuite/lib/libstdc++.exp
> +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp
> @@ -248,7 +248,7 @@ proc libstdc++_init { testfile } {
>       # Set the default timeout for v3 tests.
>       # You can override this in ~/.dejagnurc or a .exp file named by $DEJAGNU.
>       if {![info exists tool_timeout]} {
> -      set tool_timeout 600
> +      set tool_timeout 360
>       }
>
>       # Default settings.
>