From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by sourceware.org (Postfix) with ESMTPS id 5A9FE385CC8B for ; Fri, 15 Sep 2023 23:04:17 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5A9FE385CC8B Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694819057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ee0+X6boJpTEo0fCfhWbyJf44KplTcgFN0uaegykg7Y=; b=eFXcHfTsZfa1rQ3tu9xHncnX3rChA5OcCzueL9v+IB0PhraTciMxqY84FsOxK9XtH4JckP dlQXTRtDjyd5YAQzzkVozuFobjho3pCL+vBSf+NK1djy9p/pP55rJ3QWC23Wy3M/tCJAct xUlsxkwZQ6lp8f086TQf4rJL1LrgNNU= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-508-pCDyUUpYMQmgVutdDHK0iw-1; Fri, 15 Sep 2023 19:04:15 -0400 X-MC-Unique: pCDyUUpYMQmgVutdDHK0iw-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2bfe5e88d85so4396061fa.0 for ; Fri, 15 Sep 2023 16:04:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694819054; x=1695423854; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ee0+X6boJpTEo0fCfhWbyJf44KplTcgFN0uaegykg7Y=; b=aVlhsMmZacrBs8V+MxN3M8DcAJGSnNfbAF/paZ9V1nMFWf4phkawZfen2ldfQhX9Ok e++une2pXjQzrgrMS0anq925PPPc8M/FZNRoqfV9a4j02nIfw+5hYbT0J9OsbGcTgWJx oqwRewqQ6p7M2RXeboOuHvTvK4BJQTMPfdxAf6KrSAhYlGskAQxER0GLmRdAYP5/IXLx 9/+o4Ww4hULUGWQSxocRO4Z1gohPRuDESGHx0kajtgf7uhQiUue1Qnj2WC3UezfaJC1O EHoP+thWSHZj5kUBqctMaGxnzTqSfsUbcVOCh+iT7tP6MGg6S9fzih2PDQro+nqxNygz weAw== X-Gm-Message-State: AOJu0YwlZKhx00ymhfVvghA9iydNU1YIKswnEpr4VBLpSJfddver9Y8b JG/0eINtk99hOBR1oObtR76PW2XX1ncp17gRkj4Ebpps/jZzs9leTI5kjGJOyVPk0AbVVvNelA2 x7Ir4fwhSp9DguuovzHuZk256PxNUphO72oMTZA4= X-Received: by 2002:a2e:8ec1:0:b0:2bc:fffc:7f55 with SMTP id e1-20020a2e8ec1000000b002bcfffc7f55mr2720743ljl.6.1694819053952; Fri, 15 Sep 2023 16:04:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEsPcYF9ddof+/UdoZInN8tx/yejhJsWdAtp1FytqKHj2Qq3cj2J1GpuL87ELMAZR7ZimyXhi5pqnYmDvjS86I= X-Received: by 2002:a2e:8ec1:0:b0:2bc:fffc:7f55 with SMTP id e1-20020a2e8ec1000000b002bcfffc7f55mr2720732ljl.6.1694819053645; Fri, 15 Sep 2023 16:04:13 -0700 (PDT) MIME-Version: 1.0 References: <20230911163534.1913512-1-jwakely@redhat.com> In-Reply-To: <20230911163534.1913512-1-jwakely@redhat.com> From: Jonathan Wakely Date: Sat, 16 Sep 2023 00:04:02 +0100 Message-ID: Subject: Re: [PATCH 00/13] libstdc++: Add support for running tests with multiple -std options To: Jonathan Wakely Cc: libstdc++@gcc.gnu.org, gcc-patches@gcc.gnu.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-6.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Mon, 11 Sept 2023 at 17:37, Jonathan Wakely via Libstdc++ wrote: > > This patch series replicates the behaviour of the g++ testsuite, so that > libstdc++ tests can easily be run for multiple different -std options in > a single testsuite run. As described in the updated docs, the -std > options to use for every test can be overridden by setting v3_std_list > in ~/.dejagnurc or $DEJAGNU, or setting $GLIBCXX_TESTSUITE_STDS in the > environment. If not overridden, the default is just to run with > -std=gnu++17 (so that we don't increase the time taken for a full > testsuite run). > > Tests that require a newer standard than C++17 will default to that > newer standard and C++26, so e.g. std::format tests will be run with > both -std=gnu++20 and -std=gnu++26. This does increase the number of > tests, but only for the subset of tests for C++20/23/26 features. If > this is too costly for testers, we can change that (this might be > needed, because the C++20 tests for std::ranges and std::format are > particularly slow to compile). > > Because a correct default will be chosen for tests that require > something newer than C++17, we no longer need dg-options "-std=gnu++20" > or similar in any tests. Removing the explicit -std option allows the > test to be run for later standards via the v3_std_list settings, so that > we can verify that C++20 features still work in C++23 and C++26, for > example. This change already found some tests which failed when run > with a later standard (see r14-3771-gf12e26f3496275). > > Patches 2-13 in the series remove those unnecessary dg-options from > about half the relevant tests, but there are more than 500 others that > still need adjusting. > > We can remove files like testsuite/std/format/functions/format_c++23.cc > which only exist to duplicate existing tests with a different -std > option. We can remove that file now, and rely on format.cc being run > with multiple -std options by libstdc++ maintainers. > > It might also be useful to add a 'make check-quick' target which runs a > small subset of smoke tests with every standard version in v3_std_list. > This would be a suitable target for CI bots and for packagers who want > to verify that a build of GCC is functional, without running the entire > libstdc++ testsuite. This has been pushed to trunk now.