Re: [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard

[Jason Merrill <jason@redhat.com>]

On 10/19/22 11:27, Jakub Jelinek wrote:
> Hi!
> 
> On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote:
>>> The screw-up on my side with libstdc++ testing (tested normally rather
>>> than in C++23 mode) makes me wonder if we couldn't tweak the default
>>> testing.
>>> Dunno what libstdc++ testing normally does (just C++17?), make check-g++
>>> tests by default { 98, 14, 17, 20 } (and I regularly use
>>> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
>>> cover libstdc++ I guess).
>>> When adding tests for upcoming C++ version, one always has a dilemma
>>> whether to use explicit // { dg-options "-std=c++2b" }
>>> or -std=gnu++2b and similar, then the test works in all modes, but it might
>>> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
>>> test so that when 23 is tested by default and say 26 or 29 appears too,
>>> we test it also in those modes, or just go with
>>> // { dg-do whatever { target c++23 } }
>>> which has the disadvantage that it is skipped when testing by default and
>>> one only tests it if he asks for the newer version.
>>>
>>> I wonder if we couldn't for the default testing (when one doesn't
>>> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
>>> improve things a little bit by automatically treat those
>>> // { dg-do whatever { target c++23 } }
>>> tests as // { dg-options "-std=c++2b" }.
>>
>> That would be great.
> 
> Here is a patch that implements it.
> On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C
> the vanilla vs. patched gcc difference is:
>                  === g++ Summary ===
> 
> # of expected passes            1743
> # of expected failures          2
> # of unsupported tests          224
> vs.
>                  === g++ Summary ===
> 
> # of expected passes            1829
> # of expected failures          2
> # of unsupported tests          112
> 
> (g++.sum details attached).

OK, thanks.

> 2022-10-19  Jakub Jelinek  <jakub@redhat.com>
> 
> 	* lib/g++-dg.exp (g++-dg-runtest): When using defaulted
> 	std_list, if test has { dg-do * { target c++23 } } directive,
> 	use { 23 } with which the test will run instead of { 98 14 17 20 }
> 	which would make it UNSUPPORTED in all cases.
> 
> --- gcc/testsuite/lib/g++-dg.exp.jj	2022-03-30 09:11:53.306103969 +0200
> +++ gcc/testsuite/lib/g++-dg.exp	2022-10-19 17:10:51.574866364 +0200
> @@ -53,7 +53,16 @@ proc g++-dg-runtest { testcases flags de
>   	    if { [llength $gpp_std_list] > 0 } {
>   		set std_list $gpp_std_list
>   	    } else {
> -		set std_list { 98 14 17 20 }
> +		# If the test requires a newer C++ version than which
> +		# is tested by default, use that C++ version for that
> +		# single test.  This should be updated or commented
> +		# out whenever the default std_list is updated or newer
> +		# C++ effective target is added.
> +		if [search_for $test "{ dg-do * { target c++23 } }"] {
> +		    set std_list { 23 }
> +		} else {
> +		    set std_list { 98 14 17 20 }
> +		}
>   	    }
>   	    set option_list { }
>   	    foreach x $std_list {
> 
> 
> 	Jakub