From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id A35713858D39 for ; Wed, 19 Oct 2022 08:54:28 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org A35713858D39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666169668; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9BRR75CgOEJ7V1gb0wAmOx8VYwT/U69w+lFC76bxpEw=; b=I/DNTPvb8KN/Mir6Rhg5gNiAD87A1AEap72sOtDRRF6+KEdluyJvg+xwZLkYlVmI9QSJ0A IeepmZh5eo751tvnlJgle/0sopkucBNNzj641C9ExtQ5Mm+lND4cTXY+swDkmC/pGvlDUw nj0/87i+JC0IC4dwbaPncMr4hJBA1vg= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-47-q2bQpeoDNYGj89hvj8JhgQ-1; Wed, 19 Oct 2022 04:54:21 -0400 X-MC-Unique: q2bQpeoDNYGj89hvj8JhgQ-1 Received: by mail-qt1-f198.google.com with SMTP id k12-20020ac8474c000000b0039cdd680ccaso8529704qtp.1 for ; Wed, 19 Oct 2022 01:54:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9BRR75CgOEJ7V1gb0wAmOx8VYwT/U69w+lFC76bxpEw=; b=7pMmER/1txk6KVyd5XMQSo2ode52zvXyktREJXqLjhBLsnFN9W6O6DF5mEY7iVRYFZ g7vZ0/2VQcPoFXcK6zf9Okq63JG7nulvEvs9N+nTj9ZD/GKK2OA6CvJ6CMpK4aV7VR/i Fz4u/9wNPWJXR2wdyiAUP3N5RyJabnMKxFsc21RsFwWdvw24TFPRoeNP6TXiz383qdVc 09YO3dEOZcaOeqpq4JUloAVlO/ZmQLhFF/RPg4xIf9i50AiIF8boIt2mCtOrkG9wmGtC iTA0p17ZlMcLhx8SoCIcog7PAG7c4V5/rzbNlWJrWU1vPF7CBQx+VbvPBmj/dLNQqqjR HuXw== X-Gm-Message-State: ACrzQf1Y4FsmMv0cDi8OjdabwBdOd6Bz5q0wfpru2BQ4ip0g/sAtvlAU yiSotYKKGGN3a7bD0SuZ9tee40MhnsZnLL1Xs15/+OiZuOU60/xkJRJnRVX/W2GlNEeIFsiu2X0 gizcEMb8sONowDe7TkdMECaE= X-Received: by 2002:a05:6214:29ce:b0:4b4:b18:d52a with SMTP id gh14-20020a05621429ce00b004b40b18d52amr5486686qvb.52.1666169660577; Wed, 19 Oct 2022 01:54:20 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6VA4KC014/I5tLM9mz9rkFEIFVC53XcsAsj+r5x5AuQUpSQimV3cTeGijnSUHSwWWDPUUu7o6H6/DL3oSn18M= X-Received: by 2002:a05:6214:29ce:b0:4b4:b18:d52a with SMTP id gh14-20020a05621429ce00b004b40b18d52amr5486681qvb.52.1666169660334; Wed, 19 Oct 2022 01:54:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Jonathan Wakely Date: Wed, 19 Oct 2022 09:54:09 +0100 Message-ID: Subject: Re: [RFC] c++, libstdc++: Default make check vs. tests for newest C++ standard To: Jakub Jelinek Cc: Jason Merrill , gcc@gcc.gnu.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-6.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Wed, 19 Oct 2022 at 09:40, Jakub Jelinek wrote: > > Hi! > > The screw-up on my side with libstdc++ testing (tested normally rather > than in C++23 mode) makes me wonder if we couldn't tweak the default > testing. > Dunno what libstdc++ testing normally does (just C++17?), That's the default unless a test has something else in -std=gnu++17 but I do my local testing with: set target_list { "unix{,-D_GLIBCXX_USE_CXX11_ABI=0,-std=gnu++2b,-std=gnu++11}" } and then push to the compile farm and test with: set target_list { "unix{,-std=c++98,-std=gnu++11,-std=gnu++20,-D_GLIBCXX_USE_CXX11_ABI=0/-D_GLIBCXX_DEBUG,-D_GLIBCXX_DEBUG,-std=gnu++23}" } That's far too slow to force on everybody though. > make check-g++ > tests by default { 98, 14, 17, 20 } (and I regularly use > GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't > cover libstdc++ I guess). It doesn't, correct. It's been on my TODO list for a couple of years. > When adding tests for upcoming C++ version, one always has a dilemma > whether to use explicit // { dg-options "-std=c++2b" } > or -std=gnu++2b and similar, then the test works in all modes, but it might > be forgotten later on to be converted into // { dg-do whatever { target c++23 } } > test so that when 23 is tested by default and say 26 or 29 appears too, > we test it also in those modes, or just go with > // { dg-do whatever { target c++23 } } > which has the disadvantage that it is skipped when testing by default and > one only tests it if he asks for the newer version. The convention is: // { dg-options "-std=gnu++23" } // { dg-do whatever { target c++23 } } When that becomes the default, we'll remove the first line, so that it runs for all later versions. See r12-678 to r12-686 which removed the dg-options "-std=gnu++17" after that became the default for g++. I should have noticed you were missing that from some of the new tests, sorry. I saw it in a few and didn't check them all. > I wonder if we couldn't for the default testing (when one doesn't > specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar) > improve things a little bit by automatically treat those > // { dg-do whatever { target c++23 } } > tests as // { dg-options "-std=c++2b" }. > > g++-dg.exp has: > # If the testcase specifies a standard, use that one. > # If not, run it under several standards, allowing GNU extensions > # if there's a dg-options line. > if ![search_for $test "-std=*++"] { > if [search_for $test "dg-options"] { > set std_prefix "-std=gnu++" > } else { > set std_prefix "-std=c++" > } > > # See g++.exp for the initial value of this list. > global gpp_std_list > if { [llength $gpp_std_list] > 0 } { > set std_list $gpp_std_list > } else { > set std_list { 98 14 17 20 } > } > set option_list { } > foreach x $std_list { > # Handle "concepts" as C++17 plus Concepts TS. > if { $x eq "concepts" } then { set x "17 -fconcepts" > } elseif { $x eq "impcx" } then { set x "23 -fimplicit-constexpr" } > lappend option_list "${std_prefix}$x" > } > } else { > set option_list { "" } > } > > set nshort [file tail [file dirname $test]]/[file tail $test] > > foreach flags_t $option_list { > verbose "Testing $nshort, $flags $flags_t" 1 > dg-test $test "$flags $flags_t" ${default-extra-flags} > } > so I wonder if in the set std_list { 98 14 17 20 } spot we couldn't do > something like special search_for for "{ dg-do * { target c++23 } }" > and if so, set std_list { 2b } instead of set std_list { 98 14 17 20 }? > It wouldn't handle more complex cases like > // { dg-do compile { target { c++23 && { aarch64*-*-* powerpc64le*-*-linux* riscv*-*-* s390*-*-* sparc*-*-linux* } } } } > but at least for the majority of tests for the new language version > it would run them even in default testing where they'd be otherwise > skipped (we'd cycle over 98 14 17 20 only to see it doesn't satisfy any of > them). > If we wanted to go even further, we could handle similarly say c++11_only > tests. > > What do you think? But libstdc++ doesn't use g++.exp so we need to start using that (or something like it) in libstdc++ before any such changes would help.