Re: Test suite results what they mean and how to evaluate

[Ian Lance Taylor <iant@google.com>]

On Fri, Jan 10, 2014 at 7:49 AM, Mark Henwood <mhenwood@ieee.org> wrote:
>
> I have just finished building gcc 4.8.2 so that I can setup a cross
> compilation environment.  Running the test suite generated a few errors and
> failures but I am not sure how to evaluate them.  Here are the results I
> got:
>
> gcc.log:ERROR: Failed to mmap
> gcc.log:FAIL: varl is -1, not 6
> g++.log:ERROR: Failed to mmap
> g++.log:FAIL: varl is -1, not 6
> g++.log:FAIL: g++.old-deja/g++.mike/temp.C -std=c++98 execution test
> g++.log:FAIL: g++.old-deja/g++.mike/temp.C -std=c++11 execution test
> libmudflap.log:FAIL: libmudflap.c++/pass41-frag.cxx ( -O) execution test
> libmudflap.log:FAIL: libmudflap.c++/pass41-frag.cxx (-O2) execution test
> libmudflap.log:FAIL: libmudflap.c++/pass41-frag.cxx (-O3) execution test



> I am not as sure of the others:
>
> Result                           Test
> ------                           -----
> Failed to mmap
> .../gcc/testsuite/c-c++-common/asan/rlimit-mmap-test-1.c
> varl is -1
> .../gcc/testsuite/gcc.dg/guality/guality.exp
> ...c++98 execution tests.  g++.old-deja/g++.mike/temp.C -std=c++98 execution
> test
> ...c++11 execution tests.  g++.old-deja/g++.mike/temp.C -std=c++11 execution
> test
>
> My questions are
> 1. Are these failures a problem?
> 2. How can I find out what a particular test is for?
> 3. Should I submit my results to the buildstat people?


Ignore all libmudflap failures.  You aren't going to use it, and it's
going away.

Ignore all guality errors, they are very sensitive to the exact
version of gdb you happen to have installed.

The other errors are worth investigating.  I would not expect to see
them in a released version of GCC.  They don't necessarily indicate a
real problem, but you should check to make sure.

The name of the test failure is a file under gcc/testsuite.  Look at
that file and see if it does anything that might be expected to fail
in your environment.  For example, g++.old-deja/g++.mike/temp.C
expects getenv to return something meaningful.  You mentioned you have
a cross-compilation environment; perhaps getenv doesn't do what the
test expects.  Similarly the "failed to mmap" error may be a result of
your environment, in which case you may be able to ignore the test
failure.

Ian