From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by sourceware.org (Postfix) with ESMTPS id AAF7838582AC for ; Tue, 7 Nov 2023 10:37:23 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org AAF7838582AC Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=redhat.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org AAF7838582AC Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699353446; cv=none; b=FyagdFemotJGaLvF7tdl6mx00Q9F4nxdKS63OyaoKAUDpoew0XygSllhm5X4fbBeinn0KldcSvqZMsb6WefKh3q8qdeqsDRNIhF7gnh6LGQxX6RJXaqMLvDupP6Q1nuDWgrU+KoYSI9Fwu8SGyhH3sQIXgSsEOO6JUAHX++YWhE= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1699353446; c=relaxed/simple; bh=JY1n/UE1Va+omZnu0B8KnlN80kxXQJ7a+xFS15DQNPQ=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=qTjLLZ901uG5vMio4SwqyNg7B25gUJ0+FD8xBWuzKS8gq0E9OrkGL+8J5WKjr9QdzCI9aSNvctIXx/bw9OAyFsrcr8cAuODNhW07JIYP8HqPykyDWs0BsM1puo1BpmhL7+ABkVZMaYyKBRBIZFPrO40d7wXICbtVzokiccdMqLY= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1699353443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MAf93GrMig429lfm2rsvpmCx3Imnd7OlWz2iOgypB+0=; b=Fe7xrVrYNAe6TtvVM7Ky/P0Ry8CF1+vAl6fC9CrbPhtD3MCizVaHBlyOGDZyMKFbJmsh+q 7lhSQ/EvosNCLbTBmPTWGxbuXB8hQ/eR9fLt3wIdID2UABwKDkEYJtGq3z22zD+PQYALA3 V8oVlaImcnkQY5oo1SmeXldzjX9UcFA= Received: from mail-yw1-f200.google.com (mail-yw1-f200.google.com [209.85.128.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-537-DjsJNQOZN5W4CUU79mol9A-1; Tue, 07 Nov 2023 05:37:22 -0500 X-MC-Unique: DjsJNQOZN5W4CUU79mol9A-1 Received: by mail-yw1-f200.google.com with SMTP id 00721157ae682-5af16e00fadso76134437b3.0 for ; Tue, 07 Nov 2023 02:37:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699353441; x=1699958241; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MAf93GrMig429lfm2rsvpmCx3Imnd7OlWz2iOgypB+0=; b=ql6hg9B07xRhuyeiKrPOwsqdYKLO9yMepfTh8FtyE5yuZTXasQYhtbOoFaKYSGvTle nsn90mRKJu2a1BTYOdBTeQjLko/qLVnak9A0UTOH6gpOajAxUM6aYODmRQNscspq0Pi0 dvl4bSWVaZWS7X9WTst3hGLNzPZLq7g3PjTa9Kd1Ks/18bJPoElnF2B7wf9OieEB+fxv UdbpbxogMpvzhYbL22gAW93jndXwFCAvbhJNq0GNhGn+kYY3yS4j4bJGiYgGiJpIO0r/ krFwyEhjz8t0G+AaQzyKCPzOX9K0+ve9G3aqgB9vYtC28o2on8Bfc7IUp2HziePcnPm1 Fk3g== X-Gm-Message-State: AOJu0Yxaruuhi9/wp+t6yrb/t2zwxXEy4T784Y+AZEHDVREeNb10D9qf uaeKgwefqg4FiXntOMmjrYPRXNAhbc3T+llMCubeAnTeRugZTh7sJggXu9AIxLeXl3rWSb6oc3L uiOnKFXXo5twWq03xOVfvtqP6YqApiTyclg== X-Received: by 2002:a0d:f643:0:b0:594:f864:8507 with SMTP id g64-20020a0df643000000b00594f8648507mr12196990ywf.51.1699353441667; Tue, 07 Nov 2023 02:37:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IGtjNiK1hT31bMkUK39mEJcHJ7bRODY2/try8EuIqqwmdHR/uSSt7WjVd3K3MI9rA1fy+nudjqiuLKGtOpf15I= X-Received: by 2002:a0d:f643:0:b0:594:f864:8507 with SMTP id g64-20020a0df643000000b00594f8648507mr12196976ywf.51.1699353441353; Tue, 07 Nov 2023 02:37:21 -0800 (PST) MIME-Version: 1.0 References: <600FDF8F-67C5-408F-BD24-9A3964A8BFA1@comcast.net> In-Reply-To: From: Jonathan Wakely Date: Tue, 7 Nov 2023 10:37:10 +0000 Message-ID: Subject: Re: testsuite: introduce hostedlib effective target To: Alexandre Oliva Cc: Mike Stump , gcc-patches@gcc.gnu.org, Rainer Orth , libstdc++@gcc.gnu.org, =?UTF-8?Q?Arsen_Arsenovi=C4=87?= X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-6.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, 7 Nov 2023 at 10:24, Jonathan Wakely wrote: > > On Tue, 7 Nov 2023 at 10:18, Jonathan Wakely wrote: > > > > On Tue, 7 Nov 2023 at 10:04, Alexandre Oliva wrote: > > > > > > [adding libstdc++@] > > > > > > On Nov 5, 2023, Mike Stump wrote: > > > > > > > Ick. > > > > > > Indeed ;-) > > > > > > > I wish there were fewer changed lines and not 1 per test > > > > case. It feels like we've painted ourselves into a corner. > > > > > > The libstdc++ testsuite took a different approach, detecting missing > > > headers (and libraries?) at error pruning time, and xfailing the tests, > > > which seems to be more in line with what you are looking for. > > > > > > That approach, though more expedient, seems more fragile to me, in that > > > an actual bug that caused headers to go missing would cause tests to be > > > silently skipped rather than fail. > > > > I don't think we XFAIL based on missing headers. We XFAIL based on a > > specific #error message in certain headers. > > > > If a header goes missing, we'll still XFAIL. > > > > > > > > I expect the set of headers, and thus of affected tests, won't by very > > > dynamic, so it's kind of a one-shot change. > > > > > > Of course new tests might be added that rely on such headers, and would > > > likely go unnoticed until someone tries them on a non-hosted libstdc++. > > > > Since GCC 13 you don't need to build a non-hosted libstdc++ to test > > it, you can just add -ffreestanding to the runtestflags. > > > > > We could alleviate this if libstdc++ headers that are not installed on > > > hosted systems issued a warning (conditional on some macro defined by > > > the testsuite, say -D_GLIBCXX_WARN_HOSTED_ONLY). > > > > That's exactly what happens (except #error not #warning) when you > > compile with -ffreestanding. > > > > > For tests aimed > > > exclusively at hosted libstdc++, we'd then use a dg directive that both > > > implied this requirement, and changed the macro definition to suppress > > > the warning. Then anyone who added a testcase that included hosted > > > headers without indicating its hostedlib requirement would get a fail > > > even when testing with a hosted libstdc++. > > > > I don't think we need to add checks for a new macro and then use that > > when testing, you can just test with -ffreestanding instead. This > > already works today. > > Ah, reading back in the thread for the context I missed, I see that > you're specifically testing a --disable-hosted-libstdcxx build. In > that case some headers really will be absent, not just > present-with-#error. But I am still not concerned about failing to > notice if a header goes unintentionally missing, because the libstdc++ > testsuite will still notice that. > > We don't prune based on "no such header" errors, so would still get > FAILs for those tests that depend on headers which are supposed to be > present for freestanding. An alternative approach for the g++ testsuite would be to provide a set of dummy headers for the non-freestanding ones, so that all the hosted-only headers are provided by the testsuite itself, but consist of a single line: #error not available in freestanding Then match on that and XFAIL. So the individual tests themselves wouldn't need the dg-skip-if added to them, they would just automatically XFAIL if they use a hosted-only header. The difficulty would be where to add those dummy headers for , etc. so that they're only found when testing a non-hosted build. Maybe libstdc++ could provide them in the build dir for the purposes of the testsuite, but not install them?