From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by sourceware.org (Postfix) with ESMTPS id 2FC193858439; Wed, 20 Dec 2023 04:40:36 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2FC193858439 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2FC193858439 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::d35 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1703047238; cv=none; b=BrHFWCd3JhJXzBBK/r8qR4Z8eeMs7y8vZM15QZb8PNyfyt/PPRA0l7bFmCfXR2YqCAe9irUPA4OOqs5/LiDaJXoVfxw/xkeepy4VyIarMFkq7zrnRHRYpO5ei+WrJQe0rUTbKNpYoGhRGO4nHiZOk0U1OC+evduJx7pS+WFJ4Fs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1703047238; c=relaxed/simple; bh=AuTqopoOQ7MnTd0zpxE33o1F7N4zOrEZ3EZSc1G+9ME=; h=DKIM-Signature:Message-ID:Date:MIME-Version:To:From:Subject; b=tGaIoG+C21ZxmuJaEdLPatx8hWDKJpdtz4Kj9VJsAhUI2oC9H5qILedAg8mDQ9Lqgh8RlaUPCFTHMxUUeB00YSNZVQ0/GCGImyIkUyZAxB3LXQ+Hq89JC/vOOadHgSsy9K0jsJaUWMuJ2WSXSM/GzbXpoJ8WOsm9FR9tUaumN+I= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-7b7aacf63c9so233841239f.2; Tue, 19 Dec 2023 20:40:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703047234; x=1703652034; darn=gcc.gnu.org; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=z6McLMdVWBMuBUQo+JZ+x9cWvrmDDgaXcZQCWiUvDHk=; b=Bz47yOzI1lnQ+iOU1cWhC0+MqOVCfBiFIrpypCTChKNULZFiQKB2Z9XtLbJwEexvvV AtC96V0Tf/jR7cFupIICBrhKe40jddZKfuXtg0rcCkVQoW4H8HOBOwphDQOmXO8/OGwR tG/q58O/qqCs+8iyzSobnyO+iR75xXaWAuQgDA2cJ3w3QjUkIPskdRNq3djmyGTu+Byb G4rC7uXau2MxQSbqWe9/YlfAspJPLu3aU2MdS0gwmPbBxkcFDdTPFQYOMb/Kj8BBvm9g uzCVWUky4DUU6+UlXvhDtwpg3tvfY9WeKatArI9+4gPgCA8SXa84Qi2Aqb+O/lijIfJy op/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703047234; x=1703652034; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=z6McLMdVWBMuBUQo+JZ+x9cWvrmDDgaXcZQCWiUvDHk=; b=cdqITcFGpxLlJXr0l/0V0a6dAmz6qSGraJY/1W67wNSkIVq4RKk9HWFJ+eebgIf1VZ Bu/hvk1dyRcCZEvJwJgRzUZxBN7im/r8E1JXUFd3pjTPfbirknJmCl2jvV+2xKHAGwh7 0ouGzcart1F2DTxILE690CBCO+YVcXyFsUtfvb1LSqQzEakMUfCQLWE/ujNnFD6GbGtE IIhGHkQOQZ/UX/aDwTQXY1YCHHOGEeMF9sFgmruzBj9/BFrYdPVzpNBINMIILH9guOE6 0S94oi+P6rii31EGXwlsd1b2Wz1hSqo/bspv+TstW6EynBhrbyVicLOFk6KEGCqPXwx3 nquw== X-Gm-Message-State: AOJu0Ywg9rvt/FtR6yngNYNC202Jgeo00MWOB86acUaelbueegtD0a9L 75NY6ZB2PABzsgu2OO7zmsbCY+P+LSs= X-Google-Smtp-Source: AGHT+IH3woQUP+iyGeC31JeRf2EWYu8+ReIRPgSqaxOvYRXvSiLZDaGLEoW76LrXULzcSOaQri71og== X-Received: by 2002:a05:6602:4f05:b0:7b7:9401:e330 with SMTP id gl5-20020a0566024f0500b007b79401e330mr12578292iob.16.1703047234457; Tue, 19 Dec 2023 20:40:34 -0800 (PST) Received: from [172.31.0.109] ([136.36.72.243]) by smtp.gmail.com with ESMTPSA id n14-20020a6bf60e000000b007b7cd549d26sm1462108ioh.46.2023.12.19.20.40.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 19 Dec 2023 20:40:34 -0800 (PST) Message-ID: <76a02442-290a-4929-b566-035753afd859@gmail.com> Date: Tue, 19 Dec 2023 21:40:32 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Alexandre Oliva Cc: "gcc-patches@gcc.gnu.org" From: Jeff Law Subject: Problems with strub tests Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: So the strub tests in c-c++-common are problematical. They get run twice, once for C, once for C++. Yet the name of the test is the same in both runs. (by the name, I mean the name emitted into the dejagnu summary and log files). Thus if you have a test in there which passes in one context, but fails in the other, comparison tools like contrib/compare_tests may erroneously report the tests as both a test which now fails, but passed before and a test which now passes but failed before. It looks like some of the strub tests are currently known to fail with C++ and are triggering this problem Ideally we'd include the c or c++ in the test name depending on which context its being run within. That would be sufficient to resolve these problems and avoid them in the future. It would also be sufficient to get all the tests to the point where their behavior is the same for both languages. Not sure if the latter is reasonably in the cards or not. If it's not likely to land soon, any change you could look at the framework for c-c++-common and get the names unique across the two times they're run? A third option would be to change the compare_tests tool to somehow distinguish between the C and C++ tests. Not sure how feasible that is. Thanks, Jeff