Re: [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard

[Jason Merrill <jason@redhat.com>]

On 10/19/22 04:40, Jakub Jelinek wrote:
> Hi!
> 
> The screw-up on my side with libstdc++ testing (tested normally rather
> than in C++23 mode) makes me wonder if we couldn't tweak the default
> testing.
> Dunno what libstdc++ testing normally does (just C++17?), make check-g++
> tests by default { 98, 14, 17, 20 } (and I regularly use
> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
> cover libstdc++ I guess).
> When adding tests for upcoming C++ version, one always has a dilemma
> whether to use explicit // { dg-options "-std=c++2b" }
> or -std=gnu++2b and similar, then the test works in all modes, but it might
> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
> test so that when 23 is tested by default and say 26 or 29 appears too,
> we test it also in those modes, or just go with
> // { dg-do whatever { target c++23 } }
> which has the disadvantage that it is skipped when testing by default and
> one only tests it if he asks for the newer version.
> 
> I wonder if we couldn't for the default testing (when one doesn't
> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
> improve things a little bit by automatically treat those
> // { dg-do whatever { target c++23 } }
> tests as // { dg-options "-std=c++2b" }.

That would be great.

> g++-dg.exp has:
>          # If the testcase specifies a standard, use that one.
>          # If not, run it under several standards, allowing GNU extensions
>          # if there's a dg-options line.
>          if ![search_for $test "-std=*++"] {
>              if [search_for $test "dg-options"] {
>                  set std_prefix "-std=gnu++"
>              } else {
>                  set std_prefix "-std=c++"
>              }
>              
>              # See g++.exp for the initial value of this list.
>              global gpp_std_list
>              if { [llength $gpp_std_list] > 0 } {
>                  set std_list $gpp_std_list
>              } else {
>                  set std_list { 98 14 17 20 }
>              }
>              set option_list { }
>              foreach x $std_list {
>                  # Handle "concepts" as C++17 plus Concepts TS.
>                  if { $x eq "concepts" } then { set x "17 -fconcepts"
>                  } elseif { $x eq "impcx" } then { set x "23 -fimplicit-constexpr" }
>                  lappend option_list "${std_prefix}$x"
>              }
>          } else {
>              set option_list { "" }
>          }
>          
>          set nshort [file tail [file dirname $test]]/[file tail $test]
> 
>          foreach flags_t $option_list {
>              verbose "Testing $nshort, $flags $flags_t" 1
>              dg-test $test "$flags $flags_t" ${default-extra-flags}
>          }
> so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do
> something like special search_for for "{ dg-do * { target c++23 } }"
> and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }?
> It wouldn't handle more complex cases like
> // { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* riscv*-*-* s390*-*-* sparc*-*-linux* } } } }
> but at least for the majority of tests for the new language version
> it would run them even in default testing where they'd be otherwise
> skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of
> them).
> If we wanted to go even further, we could handle similarly say c++11_only
> tests.
> 
> What do you think?
> 
> 	Jakub
>