public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard
@ 2022-10-19  8:40 Jakub Jelinek
  2022-10-19  8:54 ` Jonathan Wakely
  2022-10-19 13:10 ` Jason Merrill
  0 siblings, 2 replies; 3+ messages in thread
From: Jakub Jelinek @ 2022-10-19  8:40 UTC (permalink / raw)
  To: Jason Merrill, Jonathan Wakely; +Cc: gcc

Hi!

The screw-up on my side with libstdc++ testing (tested normally rather
than in C++23 mode) makes me wonder if we couldn't tweak the default
testing.
Dunno what libstdc++ testing normally does (just C++17?), make check-g++
tests by default { 98, 14, 17, 20 } (and I regularly use
GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
cover libstdc++ I guess).
When adding tests for upcoming C++ version, one always has a dilemma
whether to use explicit // { dg-options "-std=c++2b" }
or -std=gnu++2b and similar, then the test works in all modes, but it might
be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
test so that when 23 is tested by default and say 26 or 29 appears too,
we test it also in those modes, or just go with
// { dg-do whatever { target c++23 } }
which has the disadvantage that it is skipped when testing by default and
one only tests it if he asks for the newer version.

I wonder if we couldn't for the default testing (when one doesn't
specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
improve things a little bit by automatically treat those
// { dg-do whatever { target c++23 } }
tests as // { dg-options "-std=c++2b" }.

g++-dg.exp has:
        # If the testcase specifies a standard, use that one.
        # If not, run it under several standards, allowing GNU extensions
        # if there's a dg-options line.
        if ![search_for $test "-std=*++"] {
            if [search_for $test "dg-options"] {
                set std_prefix "-std=gnu++"
            } else {
                set std_prefix "-std=c++"
            }
            
            # See g++.exp for the initial value of this list.
            global gpp_std_list
            if { [llength $gpp_std_list] > 0 } {
                set std_list $gpp_std_list
            } else {
                set std_list { 98 14 17 20 }
            }
            set option_list { }
            foreach x $std_list {
                # Handle "concepts" as C++17 plus Concepts TS.
                if { $x eq "concepts" } then { set x "17 -fconcepts"
                } elseif { $x eq "impcx" } then { set x "23 -fimplicit-constexpr" }
                lappend option_list "${std_prefix}$x"
            }
        } else {
            set option_list { "" }
        }
        
        set nshort [file tail [file dirname $test]]/[file tail $test]

        foreach flags_t $option_list {
            verbose "Testing $nshort, $flags $flags_t" 1
            dg-test $test "$flags $flags_t" ${default-extra-flags}
        }
so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do
something like special search_for for "{ dg-do * { target c++23 } }"
and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }?
It wouldn't handle more complex cases like
// { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* riscv*-*-* s390*-*-* sparc*-*-linux* } } } }
but at least for the majority of tests for the new language version
it would run them even in default testing where they'd be otherwise
skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of
them).
If we wanted to go even further, we could handle similarly say c++11_only
tests.

What do you think?

	Jakub


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard
  2022-10-19  8:40 [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard Jakub Jelinek
@ 2022-10-19  8:54 ` Jonathan Wakely
  2022-10-19 13:10 ` Jason Merrill
  1 sibling, 0 replies; 3+ messages in thread
From: Jonathan Wakely @ 2022-10-19  8:54 UTC (permalink / raw)
  To: Jakub Jelinek; +Cc: Jason Merrill, gcc

On Wed, 19 Oct 2022 at 09:40, Jakub Jelinek <jakub@redhat.com> wrote:
>
> Hi!
>
> The screw-up on my side with libstdc++ testing (tested normally rather
> than in C++23 mode) makes me wonder if we couldn't tweak the default
> testing.
> Dunno what libstdc++ testing normally does (just C++17?),

That's the default unless a test has something else in -std=gnu++17
but I do my local testing with:

set target_list {
"unix{,-D_GLIBCXX_USE_CXX11_ABI=0,-std=gnu++2b,-std=gnu++11}" }

and then push to the compile farm and test with:

set target_list {
"unix{,-std=c++98,-std=gnu++11,-std=gnu++20,-D_GLIBCXX_USE_CXX11_ABI=0/-D_GLIBCXX_DEBUG,-D_GLIBCXX_DEBUG,-std=gnu++23}"
}

That's far too slow to force on everybody though.


> make check-g++
> tests by default { 98, 14, 17, 20 } (and I regularly use
> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
> cover libstdc++ I guess).

It doesn't, correct. It's been on my TODO list for a couple of years.

> When adding tests for upcoming C++ version, one always has a dilemma
> whether to use explicit // { dg-options "-std=c++2b" }
> or -std=gnu++2b and similar, then the test works in all modes, but it might
> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
> test so that when 23 is tested by default and say 26 or 29 appears too,
> we test it also in those modes, or just go with
> // { dg-do whatever { target c++23 } }
> which has the disadvantage that it is skipped when testing by default and
> one only tests it if he asks for the newer version.

The convention is:
// { dg-options "-std=gnu++23" }
// { dg-do whatever { target c++23 } }

When that becomes the default, we'll remove the first line, so that it
runs for all later versions.
See r12-678 to r12-686 which removed the dg-options "-std=gnu++17"
after that became the default for g++.

I should have noticed you were missing that from some of the new
tests, sorry. I saw it in a few and didn't check them all.


> I wonder if we couldn't for the default testing (when one doesn't
> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
> improve things a little bit by automatically treat those
> // { dg-do whatever { target c++23 } }
> tests as // { dg-options "-std=c++2b" }.
>
> g++-dg.exp has:
>         # If the testcase specifies a standard, use that one.
>         # If not, run it under several standards, allowing GNU extensions
>         # if there's a dg-options line.
>         if ![search_for $test "-std=*++"] {
>             if [search_for $test "dg-options"] {
>                 set std_prefix "-std=gnu++"
>             } else {
>                 set std_prefix "-std=c++"
>             }
>
>             # See g++.exp for the initial value of this list.
>             global gpp_std_list
>             if { [llength $gpp_std_list] > 0 } {
>                 set std_list $gpp_std_list
>             } else {
>                 set std_list { 98 14 17 20 }
>             }
>             set option_list { }
>             foreach x $std_list {
>                 # Handle "concepts" as C++17 plus Concepts TS.
>                 if { $x eq "concepts" } then { set x "17 -fconcepts"
>                 } elseif { $x eq "impcx" } then { set x "23 -fimplicit-constexpr" }
>                 lappend option_list "${std_prefix}$x"
>             }
>         } else {
>             set option_list { "" }
>         }
>
>         set nshort [file tail [file dirname $test]]/[file tail $test]
>
>         foreach flags_t $option_list {
>             verbose "Testing $nshort, $flags $flags_t" 1
>             dg-test $test "$flags $flags_t" ${default-extra-flags}
>         }
> so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do
> something like special search_for for "{ dg-do * { target c++23 } }"
> and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }?
> It wouldn't handle more complex cases like
> // { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* riscv*-*-* s390*-*-* sparc*-*-linux* } } } }
> but at least for the majority of tests for the new language version
> it would run them even in default testing where they'd be otherwise
> skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of
> them).
> If we wanted to go even further, we could handle similarly say c++11_only
> tests.
>
> What do you think?

But libstdc++ doesn't use g++.exp so we need to start using that (or
something like it) in libstdc++ before any such changes would help.


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard
  2022-10-19  8:40 [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard Jakub Jelinek
  2022-10-19  8:54 ` Jonathan Wakely
@ 2022-10-19 13:10 ` Jason Merrill
  1 sibling, 0 replies; 3+ messages in thread
From: Jason Merrill @ 2022-10-19 13:10 UTC (permalink / raw)
  To: Jakub Jelinek, Jonathan Wakely; +Cc: gcc

On 10/19/22 04:40, Jakub Jelinek wrote:
> Hi!
> 
> The screw-up on my side with libstdc++ testing (tested normally rather
> than in C++23 mode) makes me wonder if we couldn't tweak the default
> testing.
> Dunno what libstdc++ testing normally does (just C++17?), make check-g++
> tests by default { 98, 14, 17, 20 } (and I regularly use
> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't
> cover libstdc++ I guess).
> When adding tests for upcoming C++ version, one always has a dilemma
> whether to use explicit // { dg-options "-std=c++2b" }
> or -std=gnu++2b and similar, then the test works in all modes, but it might
> be forgotten later on to be converted into // { dg-do whatever { target c++23 } }
> test so that when 23 is tested by default and say 26 or 29 appears too,
> we test it also in those modes, or just go with
> // { dg-do whatever { target c++23 } }
> which has the disadvantage that it is skipped when testing by default and
> one only tests it if he asks for the newer version.
> 
> I wonder if we couldn't for the default testing (when one doesn't
> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar)
> improve things a little bit by automatically treat those
> // { dg-do whatever { target c++23 } }
> tests as // { dg-options "-std=c++2b" }.

That would be great.

> g++-dg.exp has:
>          # If the testcase specifies a standard, use that one.
>          # If not, run it under several standards, allowing GNU extensions
>          # if there's a dg-options line.
>          if ![search_for $test "-std=*++"] {
>              if [search_for $test "dg-options"] {
>                  set std_prefix "-std=gnu++"
>              } else {
>                  set std_prefix "-std=c++"
>              }
>              
>              # See g++.exp for the initial value of this list.
>              global gpp_std_list
>              if { [llength $gpp_std_list] > 0 } {
>                  set std_list $gpp_std_list
>              } else {
>                  set std_list { 98 14 17 20 }
>              }
>              set option_list { }
>              foreach x $std_list {
>                  # Handle "concepts" as C++17 plus Concepts TS.
>                  if { $x eq "concepts" } then { set x "17 -fconcepts"
>                  } elseif { $x eq "impcx" } then { set x "23 -fimplicit-constexpr" }
>                  lappend option_list "${std_prefix}$x"
>              }
>          } else {
>              set option_list { "" }
>          }
>          
>          set nshort [file tail [file dirname $test]]/[file tail $test]
> 
>          foreach flags_t $option_list {
>              verbose "Testing $nshort, $flags $flags_t" 1
>              dg-test $test "$flags $flags_t" ${default-extra-flags}
>          }
> so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do
> something like special search_for for "{ dg-do * { target c++23 } }"
> and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }?
> It wouldn't handle more complex cases like
> // { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* riscv*-*-* s390*-*-* sparc*-*-linux* } } } }
> but at least for the majority of tests for the new language version
> it would run them even in default testing where they'd be otherwise
> skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of
> them).
> If we wanted to go even further, we could handle similarly say c++11_only
> tests.
> 
> What do you think?
> 
> 	Jakub
> 


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-10-19 13:10 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-10-19  8:40 [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard Jakub Jelinek
2022-10-19  8:54 ` Jonathan Wakely
2022-10-19 13:10 ` Jason Merrill

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).