Re: [PATCH 00/13] libstdc++: Add support for running tests with multiple -std options

[Jonathan Wakely <jwakely@redhat.com>]

On Mon, 11 Sept 2023 at 17:37, Jonathan Wakely via Libstdc++
<libstdc++@gcc.gnu.org> wrote:
>
> This patch series replicates the behaviour of the g++ testsuite, so that
> libstdc++ tests can easily be run for multiple different -std options in
> a single testsuite run.  As described in the updated docs, the -std
> options to use for every test can be overridden by setting v3_std_list
> in ~/.dejagnurc or $DEJAGNU, or setting $GLIBCXX_TESTSUITE_STDS in the
> environment.  If not overridden, the default is just to run with
> -std=gnu++17 (so that we don't increase the time taken for a full
> testsuite run).
>
> Tests that require a newer standard than C++17 will default to that
> newer standard and C++26, so e.g. std::format tests will be run with
> both -std=gnu++20 and -std=gnu++26.  This does increase the number of
> tests, but only for the subset of tests for C++20/23/26 features.  If
> this is too costly for testers, we can change that (this might be
> needed, because the C++20 tests for std::ranges and std::format are
> particularly slow to compile).
>
> Because a correct default will be chosen for tests that require
> something newer than C++17, we no longer need dg-options "-std=gnu++20"
> or similar in any tests.  Removing the explicit -std option allows the
> test to be run for later standards via the v3_std_list settings, so that
> we can verify that C++20 features still work in C++23 and C++26, for
> example.  This change already found some tests which failed when run
> with a later standard (see r14-3771-gf12e26f3496275).
>
> Patches 2-13 in the series remove those unnecessary dg-options from
> about half the relevant tests, but there are more than 500 others that
> still need adjusting.
>
> We can remove files like testsuite/std/format/functions/format_c++23.cc
> which only exist to duplicate existing tests with a different -std
> option.  We can remove that file now, and rely on format.cc being run
> with multiple -std options by libstdc++ maintainers.
>
> It might also be useful to add a 'make check-quick' target which runs a
> small subset of smoke tests with every standard version in v3_std_list.
> This would be a suitable target for CI bots and for packagers who want
> to verify that a build of GCC is functional, without running the entire
> libstdc++ testsuite.

This has been pushed to trunk now.