From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from arjuna.pair.com (arjuna.pair.com [209.68.5.131]) by sourceware.org (Postfix) with ESMTPS id 124583858C31; Sun, 7 Jan 2024 01:51:59 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 124583858C31 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=bitrange.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=bitrange.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 124583858C31 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=209.68.5.131 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704592327; cv=none; b=ZSTR55HUnFPdPwas+Q1Qtyya97mnzkrz6AkGB9QWxCEqVCbcdxwgRE3lI/n45v87uz9PO3rTp+oICdgial6T1PHiJ6C/IOaIZD5Q4/Elt9Uuoyx5C21Cn7cc8+1f7sWZb+iHjyO9o9Oly8tgOMZ2z3NDf6036EPBjOsUS8BlwMc= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1704592327; c=relaxed/simple; bh=Dkic56SDKdBo/6MJUsFIYI8GUpEfveFPV0ougC5lT9c=; h=Date:From:To:Subject:Message-ID:MIME-Version; b=cNCw+btktg58sxdgXO9Vd2a0Df4gbNkGIoVQzUoGsmBd9jZxbUw65RvlbrZeQrPOBCPs19Ey3NW3/2lEctK6ZXZKsERT8pkNeDNpxzlRSMLDBjSRUa2sB8wWX8/ZkeyZZeztf4QEsTbZzHMaAIS++aIiCIvDnr97E2K3pxHKIFE= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by arjuna.pair.com (Postfix, from userid 3006) id 80ACF8A749; Sat, 6 Jan 2024 20:51:58 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by arjuna.pair.com (Postfix) with ESMTP id 7FED68A741; Sat, 6 Jan 2024 20:51:58 -0500 (EST) Date: Sat, 6 Jan 2024 20:51:58 -0500 (EST) From: Hans-Peter Nilsson X-X-Sender: hp@arjuna.pair.com To: Jeff Law cc: Alexandre Oliva , "gcc-patches@gcc.gnu.org" Subject: Re: Problems with strub tests In-Reply-To: <76a02442-290a-4929-b566-035753afd859@gmail.com> Message-ID: References: <76a02442-290a-4929-b566-035753afd859@gmail.com> User-Agent: Alpine 2.20.16 (BSF 172 2016-09-29) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Scanned-By: mailmunge 3.11 on 209.68.5.131 X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,KAM_DMARC_STATUS,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On Tue, 19 Dec 2023, Jeff Law wrote: > > So the strub tests in c-c++-common are problematical. They get run twice, > once for C, once for C++. Yet the name of the test is the same in both runs. > (by the name, I mean the name emitted into the dejagnu summary and log files). > > Thus if you have a test in there which passes in one context, but fails in the > other, comparison tools like contrib/compare_tests may erroneously report the > tests as both a test which now fails, but passed before and a test which now > passes but failed before. > > It looks like some of the strub tests are currently known to fail with C++ and > are triggering this problem > > > Ideally we'd include the c or c++ in the test name depending on which context > its being run within. That would be sufficient to resolve these problems and > avoid them in the future. It would also be sufficient to get all the tests to > the point where their behavior is the same for both languages. > > Not sure if the latter is reasonably in the cards or not. If it's not likely > to land soon, any change you could look at the framework for c-c++-common and > get the names unique across the two times they're run? > > A third option would be to change the compare_tests tool to somehow > distinguish between the C and C++ tests. Not sure how feasible that is. How about including the name of the .sum file in the key? (They're gcc.sum and g++.sum thus different. This is what contrib/regression/btest-gcc.sh does. On the other hand, that prunes the name of the test at the first space. Don't copy that bit. :) Also not sure how feasible that is. brgds, H-P