From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx07-00178001.pphosted.com (mx07-00178001.pphosted.com [185.132.182.106]) by sourceware.org (Postfix) with ESMTPS id E12B53955CA0 for ; Wed, 16 Nov 2022 07:59:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E12B53955CA0 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=foss.st.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=foss.st.com Received: from pps.filterd (m0288072.ppops.net [127.0.0.1]) by mx07-00178001.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 2AG7vQcw023711; Wed, 16 Nov 2022 08:59:19 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=foss.st.com; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding; s=selector1; bh=Ig8+H2HCRYQKwCCTTrZj8QibX+lUoV88XObOQZ/ci48=; b=ynisnxH5hMdRX5HK2AADioW2mdPttfkfT29YbLiKhz/SyrLy4hAtDhTCTnB1+BfGIqPn 0KFp1G86S9OsI1cYkiPGdSBMW5sHkn+t/qHAtSx2xj0G6OKmNRAKHMUDwf3F6NRtPlTL O6z+vr0oTcHnvcCTgQwO0V0GpHiSZxkqaQKS9pnjJvqZ2E9eiJJxtYLGkNeyxVL/HhJ3 9OhCEIn3kTZf5/JGQwcGx0WzcE6a+R1dXopCCHCheyhTtPgex5SWgjHsjZHZktsmRilQ UA3aSebkT3wMve9cOrfw50rwt86UIrrY8v3Eh6RWfLGfOfBbLzl5lwVuInvYapQ6qYo5 zA== Received: from beta.dmz-eu.st.com (beta.dmz-eu.st.com [164.129.1.35]) by mx07-00178001.pphosted.com (PPS) with ESMTPS id 3kv94cx983-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 16 Nov 2022 08:59:19 +0100 Received: from euls16034.sgp.st.com (euls16034.sgp.st.com [10.75.44.20]) by beta.dmz-eu.st.com (STMicroelectronics) with ESMTP id 3AC3210002A; Wed, 16 Nov 2022 08:59:13 +0100 (CET) Received: from Webmail-eu.st.com (shfdag1node3.st.com [10.75.129.71]) by euls16034.sgp.st.com (STMicroelectronics) with ESMTP id 2A07D214D29; Wed, 16 Nov 2022 08:59:13 +0100 (CET) Received: from [10.211.12.156] (10.211.12.156) by SHFDAG1NODE3.st.com (10.75.129.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.6; Wed, 16 Nov 2022 08:59:12 +0100 Message-ID: <48c23331-643f-c747-9c6b-09a36938a4aa@foss.st.com> Date: Wed, 16 Nov 2022 08:59:12 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: [PATCH] testsuite: Fix mistransformed gcov Content-Language: en-US To: Hans-Peter Nilsson , CC: References: <20220909125624.2561867-1-torbjorn.svensson@foss.st.com> <662abb45-dad9-f0f7-4af6-940829d648a6@foss.st.com> <54513c03-1421-b480-fc6b-2229e811396b@foss.st.com> <20221116021154.4AE372042F@pchp3.se.axis.com> From: Torbjorn SVENSSON In-Reply-To: <20221116021154.4AE372042F@pchp3.se.axis.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.211.12.156] X-ClientProxiedBy: EQNCAS1NODE4.st.com (10.75.129.82) To SHFDAG1NODE3.st.com (10.75.129.71) X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-11-15_08,2022-11-15_03,2022-06-22_01 X-Spam-Status: No, score=-11.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,GIT_PATCH_0,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: Hi, On 2022-11-16 03:11, Hans-Peter Nilsson wrote: > How was r13-2619-g34b9a03353d3fd "gcov: Respect triplet when looking > for gcov" tested? I'm having a hard time believing it was tested with > a *cross-compiler* *in-build-tree*. I think it was only tested for > the special-case of an installed cross-compiler; not even with a > native build exercising the false branch (see patch), considering that > a breaking typo ("}" vs "]") snuck by in the first revision, before > commit. I was testing this in out-of-tree test with a cross-compiler (built for arm-none-eabi) running on Windows and Linux and just noticed the failure and (wrongly) assumed that all of the cases needed the transformation call as that's how other tools were handled. The test systems used is are hosts that does not have any internet connection, so that's why I got a typo in the first patch when moving the fix to a system that did have an internet connection. Sorry for the mess I caused! Regarding the patch you propose; it looks good to me, but I'm not able to approve it. Kind regards, Torbjörn > I guess reviewers forgot to ask that, but it's really on the > submitter; it's a general requirement for patches to say how it was > tested. Usually no more is needed than "tested with a cross-compiler > for ..., no regressions" (implying running twice and comparing results > before and after the patch). > > In this case, when adjusting test-framework bits, a little more care > and mentioned details about how it was tested, would have been in > order. It's likely it'd have shown that an uninstalled in-tree cross > (an IMHO more expected case) wasn't tested, which that patch broke for > the one gcov invocation that the testsuite does for cross-builds > (IIUC). > > It can be said like this: I tested *this* patch as follows (all > directory names below manually redacted), showing no regressions and > fixing the regression for the in-tree cross build; > > For a native build (x86_64, Debian 11): > - In the gcc build directory: > make check RUNTESTFLAGS=gcov.exp > > - In an empty new directory after native installation in /pre. > /gccsrc/top/contrib/test_installed --prefix=/pre gcov.exp > > - Also, for good measure (mentioned somewhere in the installation > documentation) native: > for tool in gcc g++ ; do env PATH=/pre/bin:$PATH runtest \ > --tool $tool --srcdir=/gccsrc/top/gcc/testsuite gcov.exp; done > > For a cris-elf cross: > - In the gcc build directory: > make check 'RUNTESTFLAGS=--target_board=cris-sim gcov.exp' > > - In an empty new directory after installation in /pre. > /gccsrc/top/contrib/test_installed --with-gcc=/pre/bin/cris-elf-gcc \ > --with-g++=/pre/bin/cris-elf-g++ --with-gfortran=/pre/bin/cris-elf-gfortran \ > --target=cris-elf --target_board=cris-sim gcov.exp > > Ok to commit? > > brgds, H-P > PS. Beware that the proc name may be up for bikeshedding. > > ---- 8< -------- 8< -------- 8< -------- 8< ---- > > In commit r13-2619-g34b9a03353d3fd, [transform] was applied to all > invocations of gcov, for both out-of-tree and in-tree testing. > For in-tree cross builds, this means gcov was called as > "/path/to/gccobj/gcc/target-tuple-gcov" gcov-pr94029.c which is > incorrect, as it's there "/path/to/gccobj/gcc/gcov" until it's > installed. This caused a testsuite failure, like: > > Running /x/gcc/gcc/testsuite/gcc.misc-tests/gcov.exp ... > FAIL: gcc.misc-tests/gcov-pr94029.c gcov failed: spawn failed > > To avoid cumbersome conditionals, use a dedicated new helper function. > > gcc/testsuite: > * lib/gcc-dg.exp (gcc-transform-out-of-tree): New proc. > * g++.dg/gcov/gcov.exp, gcc.misc-tests/gcov.exp: Call > gcc-transform-out-of-tree instead of transform. > --- > gcc/testsuite/g++.dg/gcov/gcov.exp | 4 ++-- > gcc/testsuite/gcc.misc-tests/gcov.exp | 4 ++-- > gcc/testsuite/lib/gcc-dg.exp | 13 +++++++++++++ > 3 files changed, 17 insertions(+), 4 deletions(-) > > diff --git a/gcc/testsuite/g++.dg/gcov/gcov.exp b/gcc/testsuite/g++.dg/gcov/gcov.exp > index 04e7a0164865..c9f20958836b 100644 > --- a/gcc/testsuite/g++.dg/gcov/gcov.exp > +++ b/gcc/testsuite/g++.dg/gcov/gcov.exp > @@ -24,9 +24,9 @@ global GXX_UNDER_TEST > > # Find gcov in the same directory as $GXX_UNDER_TEST. > if { ![is_remote host] && [string match "*/*" [lindex $GXX_UNDER_TEST 0]] } { > - set GCOV [file dirname [lindex $GXX_UNDER_TEST 0]]/[transform gcov] > + set GCOV [file dirname [lindex $GXX_UNDER_TEST 0]]/[gcc-transform-out-of-tree gcov] > } else { > - set GCOV [transform gcov] > + set GCOV [gcc-transform-out-of-tree gcov] > } > > # Initialize harness. > diff --git a/gcc/testsuite/gcc.misc-tests/gcov.exp b/gcc/testsuite/gcc.misc-tests/gcov.exp > index b8e9661aa537..90ceec46e0f0 100644 > --- a/gcc/testsuite/gcc.misc-tests/gcov.exp > +++ b/gcc/testsuite/gcc.misc-tests/gcov.exp > @@ -24,9 +24,9 @@ global GCC_UNDER_TEST > > # For now find gcov in the same directory as $GCC_UNDER_TEST. > if { ![is_remote host] && [string match "*/*" [lindex $GCC_UNDER_TEST 0]] } { > - set GCOV [file dirname [lindex $GCC_UNDER_TEST 0]]/[transform gcov] > + set GCOV [file dirname [lindex $GCC_UNDER_TEST 0]]/[gcc-transform-out-of-tree gcov] > } else { > - set GCOV [transform gcov] > + set GCOV [gcc-transform-out-of-tree gcov] > } > > # Initialize harness. > diff --git a/gcc/testsuite/lib/gcc-dg.exp b/gcc/testsuite/lib/gcc-dg.exp > index 23ec038f41eb..2910c9ce0998 100644 > --- a/gcc/testsuite/lib/gcc-dg.exp > +++ b/gcc/testsuite/lib/gcc-dg.exp > @@ -1429,5 +1429,18 @@ proc scan-symbol-not { args } { > } > } > > +# Transform a tool-name to its canonical-target-name by "transform" > +# (which may return the original name for native targets) but only if > +# testing out-of-tree. When in-tree, the tool is expected to be found > +# by its original name, typically with some build-directory prefix > +# prepended by the caller. > +proc gcc-transform-out-of-tree { args } { > + global TESTING_IN_BUILD_TREE > + if { [info exists TESTING_IN_BUILD_TREE] } { > + return $args; > + } > + return [transform $args] > +} > + > set additional_prunes "" > set dg_runtest_extra_prunes ""