Re: # of unexpected failures 768 ?

Re: # of unexpected failures 768 ?

[Dennis Clarke]

> On 10/31/2011 01:26 PM, Dennis Clarke wrote:
>> This seems blatantly wrong. At what point does one throw out the result
>> of
>> a bootstrap as not-acceptable ? With any non-zero value for "unexpected
>> failures" ?
>
> There's no simple answer.
>
> You need to look at the log to see what's going wrong, and run a few
> of the test cases.  There's probably a common cause.

I kept a full log of the whole test process and see tons of "internal
compiler error" which really scares me.

I'll see if I can find some common cause, hopefully.

Dennis



-- 
--
http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x1D936C72FA35B44B
+-------------------------+-----------------------------------+
| Dennis Clarke           | Solaris and Linux and Open Source |
| dclarke@blastwave.org   | Respect for open standards.       |
+-------------------------+-----------------------------------+

Re: # of unexpected failures 768 ?

[Jonathan Wakely]

On 31 October 2011 14:40, Dennis Clarke wrote:
>
>
> well, the first ice-cube in the log starts in the gcc tests thus :
>
>                === gcc tests ===
>
> Schedule of variations:
>    unix
>
> Running target unix
> Using /opt/bw/share/dejagnu/baseboards/unix.exp as board description file
> for target.
> Using /opt/bw/share/dejagnu/config/unix.exp as generic interface file for
> target.
> Using /opt/bw/src/GCC/gcc-4.6.2/gcc/testsuite/config/default.exp as
> tool-and-target-specific interface file.
> Running
> /opt/bw/src/GCC/gcc-4.6.2/gcc/testsuite/gcc.c-torture/compile/compile.exp
> ...
> FAIL: gcc.c-torture/compile/limits-exprparen.c  -O0  (internal compiler
> error)
>
>
> the test that failed with ice was :

That's not the test, that's the dejagnu driver script which runs a
whole bunch of tests under gcc.c-torture/compile/, including the one
that failed.

> Not too sure how to run that manually however here is the source of that
> test :

That's the first test.

Re: # of unexpected failures 768 ?

[Andrew Haley]

On 10/31/2011 02:40 PM, Dennis Clarke wrote:
> Is there a log file somewhere that shows more detail on what blew up ?

There's enough info in the log for you to run the test on
its own.

Andrew.

Re: # of unexpected failures 768 ?

[Dennis Clarke]

>>> You need to look at the log to see what's going wrong, and run a few
>>> of the test cases.  There's probably a common cause.
>>
>> Also, on two different architectures I see similarly poor results. Here
>> are the sparc results :
>>
>> http://gcc.gnu.org/ml/gcc-testresults/2011-10/msg03531.html
>
> That's not the log, so there's no info.

no no .. thats just the summary. I have the full logs also.

> An ICE is always a bug.

That was what bothered me.

well, the first ice-cube in the log starts in the gcc tests thus :

                === gcc tests ===

Schedule of variations:
    unix

Running target unix
Using /opt/bw/share/dejagnu/baseboards/unix.exp as board description file
for target.
Using /opt/bw/share/dejagnu/config/unix.exp as generic interface file for
target.
Using /opt/bw/src/GCC/gcc-4.6.2/gcc/testsuite/config/default.exp as
tool-and-target-specific interface file.
Running
/opt/bw/src/GCC/gcc-4.6.2/gcc/testsuite/gcc.c-torture/compile/compile.exp
...
FAIL: gcc.c-torture/compile/limits-exprparen.c  -O0  (internal compiler
error)


the test that failed with ice was :

# Expect driver script for GCC Regression Tests
# Copyright (C) 1993, 1995, 1997, 2003, 2007 Free Software Foundation
#
# This file is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with GCC; see the file COPYING3.  If not see
# <http://www.gnu.org/licenses/>.

# These tests come from Torbjorn Granlund's (tege@cygnus.com)
# C torture test suite, and other contributors.

# Load support procs.
load_lib gcc-dg.exp

# Initialize `dg'.
dg-init

# Main loop.
set saved-dg-do-what-default ${dg-do-what-default}
set dg-do-what-default "assemble"
gcc-dg-runtest [lsort [glob -nocomplain $srcdir/$subdir/*.\[cS\]]] "-w"
set dg-do-what-default ${saved-dg-do-what-default}

# All done.
dg-finish


Not too sure how to run that manually however here is the source of that
test :

# cat ../gcc-4.6.2/gcc/testsuite/gcc.c-torture/compile/limits-exprparen.c
#define LBR1 ( ( ( ( ( ( ( ( ( (
#define LBR2 LBR1 LBR1 LBR1 LBR1 LBR1 LBR1 LBR1 LBR1 LBR1 LBR1
#define LBR3 LBR2 LBR2 LBR2 LBR2 LBR2 LBR2 LBR2 LBR2 LBR2 LBR2
#define LBR4 LBR3 LBR3 LBR3 LBR3 LBR3 LBR3 LBR3 LBR3 LBR3 LBR3
#define LBR5 LBR4 LBR4 LBR4 LBR4 LBR4 LBR4 LBR4 LBR4 LBR4 LBR4
#define LBR6 LBR5 LBR5 LBR5 LBR5 LBR5 LBR5 LBR5 LBR5 LBR5 LBR5

#define RBR1 ) ) ) ) ) ) ) ) ) )
#define RBR2 RBR1 RBR1 RBR1 RBR1 RBR1 RBR1 RBR1 RBR1 RBR1 RBR1
#define RBR3 RBR2 RBR2 RBR2 RBR2 RBR2 RBR2 RBR2 RBR2 RBR2 RBR2
#define RBR4 RBR3 RBR3 RBR3 RBR3 RBR3 RBR3 RBR3 RBR3 RBR3 RBR3
#define RBR5 RBR4 RBR4 RBR4 RBR4 RBR4 RBR4 RBR4 RBR4 RBR4 RBR4
#define RBR6 RBR5 RBR5 RBR5 RBR5 RBR5 RBR5 RBR5 RBR5 RBR5 RBR5

int q5_var = LBR4 0 RBR4;


hrmm, that is a lot of parentheses !


Is there a log file somewhere that shows more detail on what blew up ?




-- 
--
http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x1D936C72FA35B44B
+-------------------------+-----------------------------------+
| Dennis Clarke           | Solaris and Linux and Open Source |
| dclarke@blastwave.org   | Respect for open standards.       |
+-------------------------+-----------------------------------+

Re: # of unexpected failures 768 ?

[Andrew Haley]

On 10/31/2011 02:15 PM, Dennis Clarke wrote:
> 
>> On 10/31/2011 01:26 PM, Dennis Clarke wrote:
>>> This seems blatantly wrong. At what point does one throw out the result
>>> of
>>> a bootstrap as not-acceptable ? With any non-zero value for "unexpected
>>> failures" ?
>>
>> There's no simple answer.
>>
>> You need to look at the log to see what's going wrong, and run a few
>> of the test cases.  There's probably a common cause.
> 
> Also, on two different architectures I see similarly poor results. Here
> are the sparc results :
> 
> http://gcc.gnu.org/ml/gcc-testresults/2011-10/msg03531.html

That's not the log, so there's no info.

An ICE is always a bug.

Andrew.

Re: # of unexpected failures 768 ?

[Dennis Clarke]

> On 10/31/2011 01:26 PM, Dennis Clarke wrote:
>> This seems blatantly wrong. At what point does one throw out the result
>> of
>> a bootstrap as not-acceptable ? With any non-zero value for "unexpected
>> failures" ?
>
> There's no simple answer.
>
> You need to look at the log to see what's going wrong, and run a few
> of the test cases.  There's probably a common cause.
>
> Andrew.

Also, on two different architectures I see similarly poor results. Here
are the sparc results :

http://gcc.gnu.org/ml/gcc-testresults/2011-10/msg03531.html

That is pretty ugly for gcc results.


-- 
--
http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x1D936C72FA35B44B
+-------------------------+-----------------------------------+
| Dennis Clarke           | Solaris and Linux and Open Source |
| dclarke@blastwave.org   | Respect for open standards.       |
+-------------------------+-----------------------------------+

Re: # of unexpected failures 768 ?

[Andrew Haley]

On 10/31/2011 01:26 PM, Dennis Clarke wrote:
> This seems blatantly wrong. At what point does one throw out the result of
> a bootstrap as not-acceptable ? With any non-zero value for "unexpected
> failures" ?

There's no simple answer.

You need to look at the log to see what's going wrong, and run a few
of the test cases.  There's probably a common cause.

Andrew.

# of unexpected failures 768 ?

[Dennis Clarke]

I sent this to the wrong list perhaps. In any case, here we go :

Dennis



---------------------------- Original Message ----------------------------
Subject: # of unexpected failures        768 ?
From:    "Dennis Clarke" <dclarke@blastwave.org>
Date:    Sun, October 30, 2011 19:28
To:      gcc@gcc.gnu.org
--------------------------------------------------------------------------


I'm not too sure how many things changed from 4.6.1 to 4.6.2 but I am
seeing a really large increase in the number of "unexpected failures" on
various tests.

With 4.6.1 and Solaris I was able to get reasonable results :

http://gcc.gnu.org/ml/gcc-testresults/2011-07/msg00139.html

Then if I use the resultant compiler from a 4.6.1 build I get a massive
increase in failures on both i386 and Sparc :

http://gcc.gnu.org/ml/gcc-testresults/2011-10/msg03286.html

and thus far on Sparc I see :

                === gcc Summary ===

# of expected passes            69236
# of unexpected failures        768
# of expected failures          235
# of unresolved testcases       1
# of unsupported tests          1240
/opt/bw/src/GCC/gcc-4.6.2-SunOS5.8-sparc/gcc/xgcc  version 4.6.2
(Blastwave.org Inc Thu Oct 27 11:33:20 GMT 2011)

and :

                === g++ Summary ===

# of expected passes            26251
# of unexpected failures        101
# of unexpected successes       1
# of expected failures          169
# of unresolved testcases       1
# of unsupported tests          496
/opt/bw/src/GCC/gcc-4.6.2-SunOS5.8-sparc/gcc/testsuite/g++/../../g++
version 4.6.2 (Blastwave.org Inc Thu Oct 27 11:33:20 GMT 2011)

This seems blatantly wrong. At what point does one throw out the result of
a bootstrap as not-acceptable ? With any non-zero value for "unexpected
failures" ?

Also, I see bucket loads of these :

FAIL: g++.dg/pch/wchar-1.C  -O2 -g -I. (internal compiler error)

What should I think about an "internal compiler error" ?

Dennis
( concerned in Solaris world )



-- 
--
http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x1D936C72FA35B44B
+-------------------------+-----------------------------------+
| Dennis Clarke           | Solaris and Linux and Open Source |
| dclarke@blastwave.org   | Respect for open standards.       |
+-------------------------+-----------------------------------+