From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 4EDE33858D39 for ; Wed, 19 Oct 2022 15:47:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 4EDE33858D39 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1666194444; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OOTiAer14diRm8W23Oips0RzXUFlcO9sdB+TPoPzmeU=; b=IJF0rzqr1+KF7CuPmD587Ojym/bO5MzcgX9LxwqNWbrt5owHIjLggxb8+igXwXhz15cQn+ dd8edAjNO78lW++NngpSly7eltl/LyG8KT+qQB4mw0/mUTpHQqAw8jHP4X8Rw7jqZcxTBf nmli1TT52/cu9kqpCV5orjKiBwhqtd8= Received: from mail-qt1-f198.google.com (mail-qt1-f198.google.com [209.85.160.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-118-leplW0HnN7yeIPdtg0ZJuA-1; Wed, 19 Oct 2022 11:47:22 -0400 X-MC-Unique: leplW0HnN7yeIPdtg0ZJuA-1 Received: by mail-qt1-f198.google.com with SMTP id cc18-20020a05622a411200b0039cdeb3a699so8896389qtb.23 for ; Wed, 19 Oct 2022 08:47:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OOTiAer14diRm8W23Oips0RzXUFlcO9sdB+TPoPzmeU=; b=TZIwyzTmRt3QNbi18i+s3qSxS3kfhCIQfdnwQ/EIFzZ6yvbjpdwDywsWhlg3zNT5u5 GqExclpfJjsTm3ZgPcPsS8BVK70x3JMOkuJSfAd/wV2YzCtt/kz0LTLuzPfj0wT8Dj63 sNanSC95UQ+0RDmiAXhbN2bwZNnov3zlOmAqSPe128rN5lJuPDcfI4oRNxE9YBc5TB/d jQJk6ERxjkBk4BKBbNOMnw2c6FkTWAomN9c5TYFja5Ll6XEA6x3gPpdjb5XjGi2a5pn4 6/y+3UsBHK6zJ5qT9ZYvN79aXfahp8KCvfwz5Bp5IgogNljpjE61CUvm9+HqrGezLY+l vl3w== X-Gm-Message-State: ACrzQf11y+6qDv8qsoT3DpXnV4nCPjbpwd/rhMROXBa+R0PsDLzhS7sy W1uK1NLT5TrY+VLN4MgDfUUsV2j9siYdq71WoeSwAdQuNTHXM8oVvZ1CLPZNNAs3zhHqZaOHEmB JiEDSPEZFw5Mg0ptc/g== X-Received: by 2002:a05:620a:8082:b0:6ee:e7b7:4624 with SMTP id ef2-20020a05620a808200b006eee7b74624mr6136611qkb.460.1666194441995; Wed, 19 Oct 2022 08:47:21 -0700 (PDT) X-Google-Smtp-Source: AMsMyM6Tbu7eRq37qsUbOWGNeBmq7ZGzd7g/APixz8EsfdD3wgn5ZcJLBMDsIdEofNhSccfIYUs7Kg== X-Received: by 2002:a05:620a:8082:b0:6ee:e7b7:4624 with SMTP id ef2-20020a05620a808200b006eee7b74624mr6136591qkb.460.1666194441680; Wed, 19 Oct 2022 08:47:21 -0700 (PDT) Received: from [192.168.1.101] (130-44-159-43.s15913.c3-0.arl-cbr1.sbo-arl.ma.cable.rcncustomer.com. [130.44.159.43]) by smtp.gmail.com with ESMTPSA id m11-20020a05620a290b00b006b929a56a2bsm5345600qkp.3.2022.10.19.08.47.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 19 Oct 2022 08:47:21 -0700 (PDT) Message-ID: <5ab36891-0b47-0637-e556-21b5dee2b2c4@redhat.com> Date: Wed, 19 Oct 2022 11:47:15 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.3.2 Subject: Re: [PATCH] testsuite: Default make check-g++ vs. tests for newest C++ standard To: Jakub Jelinek Cc: gcc-patches@gcc.gnu.org References: <2bfe4f0a-84cc-aae8-1834-6dff3918810a@redhat.com> From: Jason Merrill In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-6.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 10/19/22 11:27, Jakub Jelinek wrote: > Hi! > > On Wed, Oct 19, 2022 at 09:10:48AM -0400, Jason Merrill wrote: >>> The screw-up on my side with libstdc++ testing (tested normally rather >>> than in C++23 mode) makes me wonder if we couldn't tweak the default >>> testing. >>> Dunno what libstdc++ testing normally does (just C++17?), make check-g++ >>> tests by default { 98, 14, 17, 20 } (and I regularly use >>> GXX_TESTSUITE_STDS=98,11,14,17,20,2b in environment but that doesn't >>> cover libstdc++ I guess). >>> When adding tests for upcoming C++ version, one always has a dilemma >>> whether to use explicit // { dg-options "-std=c++2b" } >>> or -std=gnu++2b and similar, then the test works in all modes, but it might >>> be forgotten later on to be converted into // { dg-do whatever { target c++23 } } >>> test so that when 23 is tested by default and say 26 or 29 appears too, >>> we test it also in those modes, or just go with >>> // { dg-do whatever { target c++23 } } >>> which has the disadvantage that it is skipped when testing by default and >>> one only tests it if he asks for the newer version. >>> >>> I wonder if we couldn't for the default testing (when one doesn't >>> specify GXX_TESTSUITE_STDS or uses make check-c++-all and similar) >>> improve things a little bit by automatically treat those >>> // { dg-do whatever { target c++23 } } >>> tests as // { dg-options "-std=c++2b" }. >> >> That would be great. > > Here is a patch that implements it. > On make check-g++ RUNTESTFLAGS=dg.exp=cpp23/*.C > the vanilla vs. patched gcc difference is: > === g++ Summary === > > # of expected passes 1743 > # of expected failures 2 > # of unsupported tests 224 > vs. > === g++ Summary === > > # of expected passes 1829 > # of expected failures 2 > # of unsupported tests 112 > > (g++.sum details attached). OK, thanks. > 2022-10-19 Jakub Jelinek > > * lib/g++-dg.exp (g++-dg-runtest): When using defaulted > std_list, if test has { dg-do * { target c++23 } } directive, > use { 23 } with which the test will run instead of { 98 14 17 20 } > which would make it UNSUPPORTED in all cases. > > --- gcc/testsuite/lib/g++-dg.exp.jj 2022-03-30 09:11:53.306103969 +0200 > +++ gcc/testsuite/lib/g++-dg.exp 2022-10-19 17:10:51.574866364 +0200 > @@ -53,7 +53,16 @@ proc g++-dg-runtest { testcases flags de > if { [llength $gpp_std_list] > 0 } { > set std_list $gpp_std_list > } else { > - set std_list { 98 14 17 20 } > + # If the test requires a newer C++ version than which > + # is tested by default, use that C++ version for that > + # single test. This should be updated or commented > + # out whenever the default std_list is updated or newer > + # C++ effective target is added. > + if [search_for $test "{ dg-do * { target c++23 } }"] { > + set std_list { 23 } > + } else { > + set std_list { 98 14 17 20 } > + } > } > set option_list { } > foreach x $std_list { > > > Jakub