* [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard
[not found] ` <2bfe4f0a-84cc-aae8-1834-6dff3918810a@redhat.com>
@ 2022-10-19 15:27 ` Jakub Jelinek
2022-10-19 15:47 ` Jason Merrill
0 siblings, 1 reply; 2+ messages in thread
From: Jakub Jelinek @ 2022-10-19 15:27 UTC (permalink / raw)
To: Jason Merrill; +Cc: gcc-patches
[-- Attachment #1: Type: text/plain, Size: 2921 bytes --]
Hi!
On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote:
> > The screw-up on my side with libstdc++ testing (tested normally rather
> > than in C++23 mode) makes me wonder if we couldn't tweak the default
> > testing.
> > Dunno what libstdc++ testing normally does (just C++17?), make check-g++
> > tests by default { 98, 14, 17, 20 } (and I regularly use
> > GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
> > cover libstdc++ I guess).
> > When adding tests for upcoming C++ version, one always has a dilemma
> > whether to use explicit // { dg-options "-std=c++2b" }
> > or -std=gnu++2b and similar, then the test works in all modes, but it might
> > be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
> > test so that when 23 is tested by default and say 26 or 29 appears too,
> > we test it also in those modes, or just go with
> > // { dg-do whatever { target c++23 } }
> > which has the disadvantage that it is skipped when testing by default and
> > one only tests it if he asks for the newer version.
> >
> > I wonder if we couldn't for the default testing (when one doesn't
> > specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
> > improve things a little bit by automatically treat those
> > // { dg-do whatever { target c++23 } }
> > tests as // { dg-options "-std=c++2b" }.
>
> That would be great.
Here is a patch that implements it.
On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C
the vanilla vs. patched gcc difference is:
=== g++ Summary ===
# of expected passes 1743
# of expected failures 2
# of unsupported tests 224
vs.
=== g++ Summary ===
# of expected passes 1829
# of expected failures 2
# of unsupported tests 112
(g++.sum details attached).
2022-10-19 Jakub Jelinek <jakub@redhat.com>
* lib/g++-dg.exp (g++-dg-runtest): When using defaulted
std_list, if test has { dg-do * { target c++23 } } directive,
use { 23 } with which the test will run instead of { 98 14 17 20 }
which would make it UNSUPPORTED in all cases.
--- gcc/testsuite/lib/g++-dg.exp.jj 2022-03-30 09:11:53.306103969 +0200
+++ gcc/testsuite/lib/g++-dg.exp 2022-10-19 17:10:51.574866364 +0200
@@ -53,7 +53,16 @@ proc g++-dg-runtest { testcases flags de
if { [llength $gpp_std_list] > 0 } {
set std_list $gpp_std_list
} else {
- set std_list { 98 14 17 20 }
+ # If the test requires a newer C++ version than which
+ # is tested by default, use that C++ version for that
+ # single test. This should be updated or commented
+ # out whenever the default std_list is updated or newer
+ # C++ effective target is added.
+ if [search_for $test "{ dg-do * { target c++23 } }"] {
+ set std_list { 23 }
+ } else {
+ set std_list { 98 14 17 20 }
+ }
}
set option_list { }
foreach x $std_list {
Jakub
[-- Attachment #2: g++.sum.diff --]
[-- Type: text/plain, Size: 17493 bytes --]
--- testsuite/g++/g++.sum 2022-10-19 17:06:38.351280055 +0200
+++ testsuite/g++/g++.sum 2022-10-19 17:11:15.948537868 +0200
@@ -647,56 +647,39 @@ PASS: g++.dg/cpp23/auto-array4.C -std=g
PASS: g++.dg/cpp23/auto-array4.C -std=gnu++20 (test for warnings, line 12)
PASS: g++.dg/cpp23/auto-array4.C -std=gnu++20 at line 13 (test for warnings, line 12)
PASS: g++.dg/cpp23/auto-array4.C -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast1.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast10.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast11.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast12.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast2.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast3.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast4.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast5.C -std=c++20
+PASS: g++.dg/cpp23/auto-fncast1.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast10.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast11.C -std=c++23 (test for errors, line 19)
+PASS: g++.dg/cpp23/auto-fncast11.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast12.C -std=c++23 (test for errors, line 7)
+PASS: g++.dg/cpp23/auto-fncast12.C -std=c++23 (test for errors, line 8)
+PASS: g++.dg/cpp23/auto-fncast12.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast2.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 10)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 11)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 13)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 14)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 15)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 16)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 17)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 18)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 19)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for errors, line 20)
+PASS: g++.dg/cpp23/auto-fncast3.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast4.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast5.C -std=c++23 (test for errors, line 14)
+PASS: g++.dg/cpp23/auto-fncast5.C -std=c++23 (test for errors, line 15)
+PASS: g++.dg/cpp23/auto-fncast5.C -std=c++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C -std=c++98
UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C -std=c++14
UNSUPPORTED: g++.dg/cpp23/auto-fncast6.C -std=c++17
PASS: g++.dg/cpp23/auto-fncast6.C -std=c++20 (test for errors, line 10)
PASS: g++.dg/cpp23/auto-fncast6.C -std=c++20 (test for errors, line 11)
PASS: g++.dg/cpp23/auto-fncast6.C -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast7.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast8.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/auto-fncast9.C -std=c++20
+PASS: g++.dg/cpp23/auto-fncast7.C -std=c++23 (test for errors, line 8)
+PASS: g++.dg/cpp23/auto-fncast7.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast8.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/auto-fncast9.C -std=c++23 (test for excess errors)
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++98 (test for excess errors)
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++98 execution test
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++14 (test for excess errors)
@@ -705,10 +688,9 @@ PASS: g++.dg/cpp23/charlit-encoding1.C
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++17 execution test
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++20 (test for excess errors)
PASS: g++.dg/cpp23/charlit-encoding1.C -std=c++20 execution test
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/concepts-err1.C -std=c++20
+PASS: g++.dg/cpp23/concepts-err1.C -std=c++23 (test for errors, line 5)
+PASS: g++.dg/cpp23/concepts-err1.C -std=c++23 (test for errors, line 6)
+PASS: g++.dg/cpp23/concepts-err1.C -std=c++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/consteval-if1.C -std=gnu++98
UNSUPPORTED: g++.dg/cpp23/consteval-if1.C -std=gnu++14
UNSUPPORTED: g++.dg/cpp23/consteval-if1.C -std=gnu++17
@@ -904,60 +886,54 @@ PASS: g++.dg/cpp23/constexpr-nonlit7.C
PASS: g++.dg/cpp23/constexpr-nonlit7.C -std=c++17 (test for excess errors)
PASS: g++.dg/cpp23/constexpr-nonlit7.C -std=c++20 (test for excess errors)
PASS: g++.dg/cpp23/cplusplus.C (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/decltype1.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/decltype1.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/decltype1.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/decltype1.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/decltype2.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/decltype2.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/decltype2.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/decltype2.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision1.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision1.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision1.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision1.C -std=c++20
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for errors, line 14)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for warnings, line 16)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for errors, line 20)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for errors, line 41)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for warnings, line 48)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for errors, line 62)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for warnings, line 69)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for warnings, line 83)
+PASS: g++.dg/cpp23/decltype1.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/decltype2.C -std=c++23 (test for warnings, line 35)
+PASS: g++.dg/cpp23/decltype2.C -std=c++23 (test for warnings, line 47)
+PASS: g++.dg/cpp23/decltype2.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision1.C -std=c++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/elision2.C -std=c++98
UNSUPPORTED: g++.dg/cpp23/elision2.C -std=c++14
UNSUPPORTED: g++.dg/cpp23/elision2.C -std=c++17
PASS: g++.dg/cpp23/elision2.C -std=c++20 (test for errors, line 24)
PASS: g++.dg/cpp23/elision2.C -std=c++20 (test for errors, line 30)
PASS: g++.dg/cpp23/elision2.C -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/elision3.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision3.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision3.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision3.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision4.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision4.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision4.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision4.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision5.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision5.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision5.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision5.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision6.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision6.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision6.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision6.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/elision7.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/elision7.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/elision7.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/elision7.C -std=c++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating1.C -std=gnu++20
+PASS: g++.dg/cpp23/elision3.C -std=c++23 (test for errors, line 15)
+PASS: g++.dg/cpp23/elision3.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision4.C -std=c++23 (test for errors, line 14)
+PASS: g++.dg/cpp23/elision4.C -std=c++23 (test for errors, line 27)
+PASS: g++.dg/cpp23/elision4.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision5.C -std=c++23 (test for errors, line 34)
+PASS: g++.dg/cpp23/elision5.C -std=c++23 (test for errors, line 43)
+PASS: g++.dg/cpp23/elision5.C -std=c++23 (test for warnings, line 52)
+PASS: g++.dg/cpp23/elision5.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision6.C -std=c++23 (test for errors, line 16)
+PASS: g++.dg/cpp23/elision6.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/elision7.C -std=c++23 (test for errors, line 44)
+PASS: g++.dg/cpp23/elision7.C -std=c++23 (test for warnings, line 46)
+PASS: g++.dg/cpp23/elision7.C -std=c++23 (test for errors, line 53)
+PASS: g++.dg/cpp23/elision7.C -std=c++23 (test for errors, line 65)
+PASS: g++.dg/cpp23/elision7.C -std=c++23 (test for excess errors)
+PASS: g++.dg/cpp23/ext-floating1.C -std=gnu++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/ext-floating10.C -std=gnu++98
UNSUPPORTED: g++.dg/cpp23/ext-floating10.C -std=gnu++14
UNSUPPORTED: g++.dg/cpp23/ext-floating10.C -std=gnu++17
UNSUPPORTED: g++.dg/cpp23/ext-floating10.C -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating11.C -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating2.C -std=gnu++20
+PASS: g++.dg/cpp23/ext-floating11.C -std=gnu++23 (test for excess errors)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 55)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 59)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 63)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 81)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 85)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for warnings, line 110)
+PASS: g++.dg/cpp23/ext-floating2.C -std=gnu++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/ext-floating3.C -std=gnu++98
UNSUPPORTED: g++.dg/cpp23/ext-floating3.C -std=gnu++14
UNSUPPORTED: g++.dg/cpp23/ext-floating3.C -std=gnu++17
@@ -970,10 +946,7 @@ UNSUPPORTED: g++.dg/cpp23/ext-floating5.
UNSUPPORTED: g++.dg/cpp23/ext-floating5.C -std=gnu++14
UNSUPPORTED: g++.dg/cpp23/ext-floating5.C -std=gnu++17
UNSUPPORTED: g++.dg/cpp23/ext-floating5.C -std=gnu++20
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C -std=gnu++98
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C -std=gnu++14
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C -std=gnu++17
-UNSUPPORTED: g++.dg/cpp23/ext-floating6.C -std=gnu++20
+PASS: g++.dg/cpp23/ext-floating6.C -std=gnu++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/ext-floating7.C -std=gnu++98
UNSUPPORTED: g++.dg/cpp23/ext-floating7.C -std=gnu++14
UNSUPPORTED: g++.dg/cpp23/ext-floating7.C -std=gnu++17
@@ -996,10 +969,13 @@ PASS: g++.dg/cpp23/init-stmt1.C -std=c+
PASS: g++.dg/cpp23/init-stmt1.C -std=c++20 (test for errors, line 20)
PASS: g++.dg/cpp23/init-stmt1.C -std=c++20 (test for errors, line 25)
PASS: g++.dg/cpp23/init-stmt1.C -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/init-stmt2.C -std=c++20
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 12)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 14)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 16)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 18)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 20)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for errors, line 22)
+PASS: g++.dg/cpp23/init-stmt2.C -std=c++23 (test for excess errors)
PASS: g++.dg/cpp23/label1.C -std=c++98 (test for errors, line 14)
PASS: g++.dg/cpp23/label1.C -std=c++98 (test for errors, line 20)
PASS: g++.dg/cpp23/label1.C -std=c++98 (test for errors, line 28)
@@ -1043,10 +1019,14 @@ PASS: g++.dg/cpp23/label1.C -std=c++20
PASS: g++.dg/cpp23/label1.C -std=c++20 (test for errors, line 84)
PASS: g++.dg/cpp23/label1.C -std=c++20 (test for errors, line 88)
PASS: g++.dg/cpp23/label1.C -std=c++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/label2.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/label2.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/label2.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/label2.C -std=c++20
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 12)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 22)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 29)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 30)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 37)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 44)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for errors, line 52)
+PASS: g++.dg/cpp23/label2.C -std=c++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/lambda-attr1.C -std=c++98
PASS: g++.dg/cpp23/lambda-attr1.C -std=c++14 (test for warnings, line 7)
PASS: g++.dg/cpp23/lambda-attr1.C -std=c++14 (test for warnings, line 10)
@@ -1130,10 +1110,7 @@ PASS: g++.dg/cpp23/lambda-specifiers1.C
PASS: g++.dg/cpp23/lambda-specifiers1.C -std=gnu++20 (test for warnings, line 16)
PASS: g++.dg/cpp23/lambda-specifiers1.C -std=gnu++20 (test for warnings, line 17)
PASS: g++.dg/cpp23/lambda-specifiers1.C -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/lambda-specifiers2.C -std=c++20
+PASS: g++.dg/cpp23/lambda-specifiers2.C -std=c++23 (test for excess errors)
PASS: g++.dg/cpp23/lookup2.C -std=c++98 (test for excess errors)
PASS: g++.dg/cpp23/lookup2.C -std=c++14 (test for excess errors)
PASS: g++.dg/cpp23/lookup2.C -std=c++17 (test for excess errors)
@@ -1688,10 +1665,7 @@ PASS: g++.dg/cpp23/normalize7.C -std=gn
PASS: g++.dg/cpp23/normalize7.C -std=gnu++20 (test for errors, line 6)
PASS: g++.dg/cpp23/normalize7.C -std=gnu++20 (test for errors, line 8)
PASS: g++.dg/cpp23/normalize7.C -std=gnu++20 (test for excess errors)
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C -std=c++98
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C -std=c++14
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C -std=c++17
-UNSUPPORTED: g++.dg/cpp23/size_t-literals.C -std=c++20
+PASS: g++.dg/cpp23/size_t-literals.C -std=c++23 (test for excess errors)
UNSUPPORTED: g++.dg/cpp23/static-operator-call1.C -std=gnu++98
PASS: g++.dg/cpp23/static-operator-call1.C -std=gnu++14 (test for warnings, line 8)
PASS: g++.dg/cpp23/static-operator-call1.C -std=gnu++14 (test for warnings, line 30)
@@ -1980,8 +1954,8 @@ PASS: g++.dg/cpp23/whitespace-splice1.C
=== g++ Summary ===
-# of expected passes 1743
+# of expected passes 1829
# of expected failures 2
-# of unsupported tests 224
+# of unsupported tests 112
/usr/src/gcc/obj/gcc/xg++ version 13.0.0 20221018 (experimental) (GCC)
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard
2022-10-19 15:27 ` [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard Jakub Jelinek
@ 2022-10-19 15:47 ` Jason Merrill
0 siblings, 0 replies; 2+ messages in thread
From: Jason Merrill @ 2022-10-19 15:47 UTC (permalink / raw)
To: Jakub Jelinek; +Cc: gcc-patches
On 10/19/22 11:27, Jakub Jelinek wrote:
> Hi!
>
> On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote:
>>> The screw-up on my side with libstdc++ testing (tested normally rather
>>> than in C++23 mode) makes me wonder if we couldn't tweak the default
>>> testing.
>>> Dunno what libstdc++ testing normally does (just C++17?), make check-g++
>>> tests by default { 98, 14, 17, 20 } (and I regularly use
>>> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
>>> cover libstdc++ I guess).
>>> When adding tests for upcoming C++ version, one always has a dilemma
>>> whether to use explicit // { dg-options "-std=c++2b" }
>>> or -std=gnu++2b and similar, then the test works in all modes, but it might
>>> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
>>> test so that when 23 is tested by default and say 26 or 29 appears too,
>>> we test it also in those modes, or just go with
>>> // { dg-do whatever { target c++23 } }
>>> which has the disadvantage that it is skipped when testing by default and
>>> one only tests it if he asks for the newer version.
>>>
>>> I wonder if we couldn't for the default testing (when one doesn't
>>> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
>>> improve things a little bit by automatically treat those
>>> // { dg-do whatever { target c++23 } }
>>> tests as // { dg-options "-std=c++2b" }.
>>
>> That would be great.
>
> Here is a patch that implements it.
> On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C
> the vanilla vs. patched gcc difference is:
> === g++ Summary ===
>
> # of expected passes 1743
> # of expected failures 2
> # of unsupported tests 224
> vs.
> === g++ Summary ===
>
> # of expected passes 1829
> # of expected failures 2
> # of unsupported tests 112
>
> (g++.sum details attached).
OK, thanks.
> 2022-10-19 Jakub Jelinek <jakub@redhat.com>
>
> * lib/g++-dg.exp (g++-dg-runtest): When using defaulted
> std_list, if test has { dg-do * { target c++23 } } directive,
> use { 23 } with which the test will run instead of { 98 14 17 20 }
> which would make it UNSUPPORTED in all cases.
>
> --- gcc/testsuite/lib/g++-dg.exp.jj 2022-03-30 09:11:53.306103969 +0200
> +++ gcc/testsuite/lib/g++-dg.exp 2022-10-19 17:10:51.574866364 +0200
> @@ -53,7 +53,16 @@ proc g++-dg-runtest { testcases flags de
> if { [llength $gpp_std_list] > 0 } {
> set std_list $gpp_std_list
> } else {
> - set std_list { 98 14 17 20 }
> + # If the test requires a newer C++ version than which
> + # is tested by default, use that C++ version for that
> + # single test. This should be updated or commented
> + # out whenever the default std_list is updated or newer
> + # C++ effective target is added.
> + if [search_for $test "{ dg-do * { target c++23 } }"] {
> + set std_list { 23 }
> + } else {
> + set std_list { 98 14 17 20 }
> + }
> }
> set option_list { }
> foreach x $std_list {
>
>
> Jakub
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-10-19 15:47 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <Y0+36fGnYINE/VJN@tucnak>
[not found] ` <2bfe4f0a-84cc-aae8-1834-6dff3918810a@redhat.com>
2022-10-19 15:27 ` [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard Jakub Jelinek
2022-10-19 15:47 ` Jason Merrill
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).