Re: [PATCH] testsuite: Fix mistransformed gcov

[Richard Biener <richard.guenther@gmail.com>]

On Wed, Nov 16, 2022 at 9:00 AM Torbjorn SVENSSON via Gcc-patches
<gcc-patches@gcc.gnu.org> wrote:
>
> Hi,
>
> On 2022-11-16 03:11, Hans-Peter Nilsson wrote:
> > How was r13-2619-g34b9a03353d3fd "gcov: Respect triplet when looking
> > for gcov" tested?  I'm having a hard time believing it was tested with
> > a *cross-compiler* *in-build-tree*.  I think it was only tested for
> > the special-case of an installed cross-compiler; not even with a
> > native build exercising the false branch (see patch), considering that
> > a breaking typo ("}" vs "]") snuck by in the first revision, before
> > commit.
>
> I was testing this in out-of-tree test with a cross-compiler (built for
> arm-none-eabi) running on Windows and Linux and just noticed the failure
> and (wrongly) assumed that all of the cases needed the transformation
> call as that's how other tools were handled.
> The test systems used is are hosts that does not have any internet
> connection, so that's why I got a typo in the first patch when moving
> the fix to a system that did have an internet connection.
> Sorry for the mess I caused!
>
> Regarding the patch you propose; it looks good to me, but I'm not able
> to approve it.

The patch is OK.

Richard.

> Kind regards,
> Torbjörn
>
> > I guess reviewers forgot to ask that, but it's really on the
> > submitter; it's a general requirement for patches to say how it was
> > tested.  Usually no more is needed than "tested with a cross-compiler
> > for ..., no regressions" (implying running twice and comparing results
> > before and after the patch).
> >
> > In this case, when adjusting test-framework bits, a little more care
> > and mentioned details about how it was tested, would have been in
> > order.  It's likely it'd have shown that an uninstalled in-tree cross
> > (an IMHO more expected case) wasn't tested, which that patch broke for
> > the one gcov invocation that the testsuite does for cross-builds
> > (IIUC).
> >
> > It can be said like this: I tested *this* patch as follows (all
> > directory names below manually redacted), showing no regressions and
> > fixing the regression for the in-tree cross build;
> >
> > For a native build (x86_64, Debian 11):
> > - In the gcc build directory:
> >   make check RUNTESTFLAGS=gcov.exp
> >
> > - In an empty new directory after native installation in /pre.
> > /gccsrc/top/contrib/test_installed --prefix=/pre gcov.exp
> >
> > - Also, for good measure (mentioned somewhere in the installation
> > documentation) native:
> > for tool in gcc g++ ; do env PATH=/pre/bin:$PATH runtest \
> >   --tool $tool --srcdir=/gccsrc/top/gcc/testsuite gcov.exp; done
> >
> > For a cris-elf cross:
> > - In the gcc build directory:
> > make check 'RUNTESTFLAGS=--target_board=cris-sim gcov.exp'
> >
> > - In an empty new directory after installation in /pre.
> > /gccsrc/top/contrib/test_installed --with-gcc=/pre/bin/cris-elf-gcc \
> >   --with-g++=/pre/bin/cris-elf-g++ --with-gfortran=/pre/bin/cris-elf-gfortran \
> >   --target=cris-elf --target_board=cris-sim gcov.exp
> >
> > Ok to commit?
> >
> > brgds, H-P
> > PS. Beware that the proc name may be up for bikeshedding.
> >
> > ---- 8< -------- 8< -------- 8< -------- 8< ----
> >
> > In commit r13-2619-g34b9a03353d3fd, [transform] was applied to all
> > invocations of gcov, for both out-of-tree and in-tree testing.
> > For in-tree cross builds, this means gcov was called as
> > "/path/to/gccobj/gcc/target-tuple-gcov" gcov-pr94029.c which is
> > incorrect, as it's there "/path/to/gccobj/gcc/gcov" until it's
> > installed.  This caused a testsuite failure, like:
> >
> > Running /x/gcc/gcc/testsuite/gcc.misc-tests/gcov.exp ...
> > FAIL: gcc.misc-tests/gcov-pr94029.c gcov failed: spawn failed
> >
> > To avoid cumbersome conditionals, use a dedicated new helper function.
> >
> > gcc/testsuite:
> >       * lib/gcc-dg.exp (gcc-transform-out-of-tree): New proc.
> >       * g++.dg/gcov/gcov.exp, gcc.misc-tests/gcov.exp: Call
> >       gcc-transform-out-of-tree instead of transform.
> > ---
> >   gcc/testsuite/g++.dg/gcov/gcov.exp    |  4 ++--
> >   gcc/testsuite/gcc.misc-tests/gcov.exp |  4 ++--
> >   gcc/testsuite/lib/gcc-dg.exp          | 13 +++++++++++++
> >   3 files changed, 17 insertions(+), 4 deletions(-)
> >
> > diff --git a/gcc/testsuite/g++.dg/gcov/gcov.exp b/gcc/testsuite/g++.dg/gcov/gcov.exp
> > index 04e7a0164865..c9f20958836b 100644
> > --- a/gcc/testsuite/g++.dg/gcov/gcov.exp
> > +++ b/gcc/testsuite/g++.dg/gcov/gcov.exp
> > @@ -24,9 +24,9 @@ global GXX_UNDER_TEST
> >
> >   # Find gcov in the same directory as $GXX_UNDER_TEST.
> >   if { ![is_remote host] && [string match "*/*" [lindex $GXX_UNDER_TEST 0]] } {
> > -    set GCOV [file dirname [lindex $GXX_UNDER_TEST 0]]/[transform gcov]
> > +    set GCOV [file dirname [lindex $GXX_UNDER_TEST 0]]/[gcc-transform-out-of-tree gcov]
> >   } else {
> > -    set GCOV [transform gcov]
> > +    set GCOV [gcc-transform-out-of-tree gcov]
> >   }
> >
> >   # Initialize harness.
> > diff --git a/gcc/testsuite/gcc.misc-tests/gcov.exp b/gcc/testsuite/gcc.misc-tests/gcov.exp
> > index b8e9661aa537..90ceec46e0f0 100644
> > --- a/gcc/testsuite/gcc.misc-tests/gcov.exp
> > +++ b/gcc/testsuite/gcc.misc-tests/gcov.exp
> > @@ -24,9 +24,9 @@ global GCC_UNDER_TEST
> >
> >   # For now find gcov in the same directory as $GCC_UNDER_TEST.
> >   if { ![is_remote host] && [string match "*/*" [lindex $GCC_UNDER_TEST 0]] } {
> > -    set GCOV [file dirname [lindex $GCC_UNDER_TEST 0]]/[transform gcov]
> > +    set GCOV [file dirname [lindex $GCC_UNDER_TEST 0]]/[gcc-transform-out-of-tree gcov]
> >   } else {
> > -    set GCOV [transform gcov]
> > +    set GCOV [gcc-transform-out-of-tree gcov]
> >   }
> >
> >   # Initialize harness.
> > diff --git a/gcc/testsuite/lib/gcc-dg.exp b/gcc/testsuite/lib/gcc-dg.exp
> > index 23ec038f41eb..2910c9ce0998 100644
> > --- a/gcc/testsuite/lib/gcc-dg.exp
> > +++ b/gcc/testsuite/lib/gcc-dg.exp
> > @@ -1429,5 +1429,18 @@ proc scan-symbol-not { args } {
> >       }
> >   }
> >
> > +# Transform a tool-name to its canonical-target-name by "transform"
> > +# (which may return the original name for native targets) but only if
> > +# testing out-of-tree.  When in-tree, the tool is expected to be found
> > +# by its original name, typically with some build-directory prefix
> > +# prepended by the caller.
> > +proc gcc-transform-out-of-tree { args } {
> > +    global TESTING_IN_BUILD_TREE
> > +    if { [info exists TESTING_IN_BUILD_TREE] } {
> > +     return $args;
> > +    }
> > +    return [transform $args]
> > +}
> > +
> >   set additional_prunes ""
> >   set dg_runtest_extra_prunes ""