From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from esa2.mentor.iphmx.com (esa2.mentor.iphmx.com [68.232.141.98]) by sourceware.org (Postfix) with ESMTPS id 58D253861C53; Fri, 16 Jul 2021 18:22:41 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 58D253861C53 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mentor.com IronPort-SDR: cFniITG5I/ZrlfGM16FUkAQwRBVtLWGp+6cBgI3moSZkwfy5AEIK7Le7hlxqRDETbFbOu+s8Sx OUvuYZxhGFyIlWHUr+ICEKYJvwuhb63JxpD3VCIsPeJe5dCix+qizfSaryQgIDlTtpwt5dZE7Z tAiuCyeiTQXNtjt8Quy+iifYD1ZTT+iRSuWZQMRO0YbocN6awgPOuJBrb1XxotG/PjyCyI+jYN gmNgDYPIVPrSFuHF46Y2YhCZwGYgms7FllfRQJBmx9sJqiuvtYuH/iDVwtnu/aKXTjmqcdRyPQ QNsf2l3l66VqEM+wX30IiJ9N X-IronPort-AV: E=Sophos;i="5.84,245,1620720000"; d="scan'208";a="63614797" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa2.mentor.iphmx.com with ESMTP; 16 Jul 2021 10:22:40 -0800 IronPort-SDR: h0krFIaJNHn8FMLK12ih43w0aRWm3YMglmXCLe3x8VvcdlNMqfansn685JEgDyWYxqy5KdMXJF RxD5QMGefjd6rIvAI29rqfazx6UBFud5T/ZniD4A8bTdT7jhmF+/0Dh6tcO0hyGdAtm08uocT1 Q5Xy6ZqEMH6+FVBg/P70UoKiFbzDi12FZUQEkCRxoP7Bv6VdKgB8mLPPZm4AW2p7WhE17bWpEZ 1yg3vP5hHHPW22IgBb9beZp/ljMriXI0+6nM2kPMdn8GfQEBoESR/zc7HIQchPMJAuvBeY+ttn 5Ns= Subject: Re: Pushing XFAILed test cases To: Thomas Schwinge , Thomas Koenig , CC: , References: <13168f92-8863-cb63-9470-a6055d5da5f6@codesourcery.com> <10658f98-0a72-e80d-0cc6-7b4624eea1f1@netcologne.de> <87im1ab7g4.fsf@euler.schwinge.homeip.net> From: Sandra Loosemore Message-ID: <18f90084-4d3a-f6f0-8a2a-d305ce152b0d@codesourcery.com> Date: Fri, 16 Jul 2021 12:22:33 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <87im1ab7g4.fsf@euler.schwinge.homeip.net> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ClientProxiedBy: SVR-ORW-MBX-05.mgc.mentorg.com (147.34.90.205) To svr-orw-mbx-03.mgc.mentorg.com (147.34.90.203) X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, NICE_REPLY_A, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2021 18:22:43 -0000 On 7/16/21 9:32 AM, Thomas Schwinge wrote: > > [much snipped] > > Of course, we shall assume a certain level of quality in the XFAILed test > cases: I'm certainly not suggesting we put any random junk into the > testsuite, coarsely XFAILed. (I have not reviewed Sandra's test cases to > that effect, but knowing here, I'd be surprised if that were the problem > here.) FWIW, Tobias already did an extensive review of an early version of the testsuite patches in question and pointed out several cases where failures were due to my misunderstanding of the language standard or general confusion about what the expected behavior was supposed to be when gfortran wasn't implementing it or was tripping over other bugs. :-S I hope I incorporated all his suggestions and rewrote the previously-bogus tests to be more useful for the version I posted for review on the Fortran list, but shouldn't the normal patch review process be adequate to take care of any additional concerns about quality? My previous understanding of the development process and testsuite conventions is that adding tests that FAIL is bad, but XFAILing them with reference to a PR is OK, and certainly much better than simply not having test coverage of those things at all. Especially in the case of something like the TS29113 testsuite where the explicit goal is to track standards compliance and/or the completeness of the existing implementation. :-S So it seems to me rather surprising to take the position that we should not be committing any new test cases that need to be XFAILed. :-S -Sandra