From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from esa2.mentor.iphmx.com (esa2.mentor.iphmx.com [68.232.141.98]) by sourceware.org (Postfix) with ESMTPS id 5EA97385701A; Wed, 21 Jul 2021 09:20:43 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5EA97385701A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mentor.com IronPort-SDR: /Qin2RZPbN5xyZqbr9Mm0iw+gMFr72q6IZTZ+dxcYZNI5AJqbyvuGwkzb3zhCe3X0mRvi+JKE4 lo/OBQ2oAohSBXsO8bOmLomOonQzdhjX2mBGNBgvmBaVtscuwcw3GO02j4cEs8/e/XKE/2TAQg yQopeQO92qHG8XQRn/C7E8sFrcfi3kxzwQb6Dp1BZx5c91XMcZM9A57Z6vvzz08Dy6Z1NGNFLp s3DDIWcl/aImc+Zcf3rB7B24+kJcsBY2BBoCGEzirk51R1sZvKvW2YmQEQGyH9JudYpUmQ1mby XgIjefmyKVA+2HzxAgstETth X-IronPort-AV: E=Sophos;i="5.84,257,1620720000"; d="scan'208";a="63763743" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa2.mentor.iphmx.com with ESMTP; 21 Jul 2021 01:20:42 -0800 IronPort-SDR: f1vMRAf+zZb21LpTKX9cD5U+JH82L7a4MemBz6LXbVWbFzGmxOgS1BYF3/wIUoAeimmRtx0MUX ED8kOjsVC9XfyLUhdJgq8PUJyMtg19GUblGXdz9RbLCW8M/OezVtGkITJQAdX6j81nb7PqorX+ ve5f9brM7qSaAWqrLMGK8a1yqCpiaKXXQKPY7oZI8IBhjCcHGkeh21TFva6+5CKsNJSMDZs0iA Rqgt6vo+Pu24s5o51hJYSaXAzSH2V/sI18gMQ+3AekqfO7WaP4t7tr6rVi1ZNME2YJAFlQPyBP XwY= Subject: Re: Pushing XFAILed test cases To: Thomas Koenig , Sandra Loosemore , Thomas Schwinge , CC: , References: <13168f92-8863-cb63-9470-a6055d5da5f6@codesourcery.com> <10658f98-0a72-e80d-0cc6-7b4624eea1f1@netcologne.de> <87im1ab7g4.fsf@euler.schwinge.homeip.net> <18f90084-4d3a-f6f0-8a2a-d305ce152b0d@codesourcery.com> From: Tobias Burnus Message-ID: Date: Wed, 21 Jul 2021 11:20:35 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-09.mgc.mentorg.com (139.181.222.9) To svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) X-Spam-Status: No, score=-5.8 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, KAM_DMARC_STATUS, NICE_REPLY_A, SPF_HELO_PASS, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: gcc@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gcc mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jul 2021 09:20:45 -0000 Hi all, hi Thomas (2x), hi Sandra, On 16.07.21 09:52, Thomas Koenig via Fortran wrote: >> The part of the patch to add tests for this goes on top of my base >> TS29113 testsuite patch, which hasn't been reviewed or committed yet. > > It is my understanding that it is not gcc policy to add xfailed test > cases for things that do not yet work. Rather, xfail is for tests that > later turn out not to work, especially on certain architectures. ... On 17.07.21 09:25, Thomas Koenig via Fortran wrote: > Is it or is it not gcc policy to push a large number of test cases > that currently do not work and XFAIL them? In my opinion, it is bad to add testcases which _only_ consist of xfails for 'target *-*-*'; however, for an extensive set of test cases, I think it is better to xfail missing parts than to comment them out - or not having them at all. That permits a better test coverage once the features have been implemented. For the TS29113 patch, which Sandra has posted on July 7, I count: * 77 'dg-do run' tests - of which 27 are xfailed (35%) * 28 compile-time tests * 291 dg-error - of which 59 are xfailed (20%) * 29 dg-bogus - of which are 25 are xfailed (86%) (And of course, those lines which are valid do not have a dg-error - and usually also no dg-bogus.) And in total: * 1 '.exp' file * 105 '.f90' files (with 8232 lines in total including comment lines) * 53 '.c'files (5281 lines) * 1 '.h' file (12 lines) Hence, for me this sounds a rather reasonable amount of xfail. Especially, given that several pending patches do/will reduce the amount of xfails by fixing issues exposed by the testsuite (which has been posted but so far not reviewed). Of course, in an ideal world, xfail would not exist :-) On 07.07.21 05:40, Sandra Loosemore wrote: > There was a question in one of the issues about why this testsuite > references TS29113 instead of the 2018 standard. Well, that is what > our customer is interested in: finding out what parts of the TS29113 > functionality remain unimplemented or broken, and fixing them, so that > gfortran can say that it implements that specification. I believe the only real difference between TS29113 and Fortran 2018's interoperability support is that 'select rank' was added in Fortran 2018. The testsuite also tests 'select rank'; in that sense, it is also for Fortran 2018. Thus, ts29113 + ts29113.exp or 'f2018-c-interop' + 'f2018-c-interop.exp' are both fine to me. =E2=80=94 'ts29113' is shorter while the other is clearer to those who did not follow the Fortran standards and missed that there was a technical specification (TS) between F2008 and F2018, incorporated (with tiny modifications) in F2018. Tobias ----------------- Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstra=C3=9Fe 201= , 80634 M=C3=BCnchen; Gesellschaft mit beschr=C3=A4nkter Haftung; Gesch=C3= =A4ftsf=C3=BChrer: Thomas Heurung, Frank Th=C3=BCrauf; Sitz der Gesellschaf= t: M=C3=BCnchen; Registergericht M=C3=BCnchen, HRB 106955