Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[poschmid]

I ran many tests on my i686-pc-linux-gnu computer, the CPU is
a Pentium IV with a 1.8 GHz clock. 

I tested the following applications: Blitz++-0.7, FTensor-1.1pre25,
STLport-4.6.1, STLport-5.0-0409, qt-x11-free-3.3.0, root_v3.05.01,
root_v3.10.02, root_v4.00.02, cln-1.1.5, ACE-5.4+TAO-1.4+CIAO-0.4, stepanov,
oopack, mtl-2.1.2-21, estl, bench++, boost, the code examples that are
included in the books by Breymann, Josuttis and Vandevoorde and other 
applications.

After applying some patches of my own, the remaining problems are:

1)qt-x11-free-3.3.0 does not build (see PR 14400). This problem can be worked
around by configuring with -system-zlib -system-libpng enabled. Although the
build completes when configured this way, many applications crash on shutdown
with a segmentation fault.
2) pooma-gcc does not build (see PR 14545).  
3) There are 10 failures while running the current boost regression test
suite. All of them might be regressions since gcc 3.3 accepts the code which
is rejected by gcc 3.4.  
4) Some test examples of ACE and TAO fail.  
5) Because of recent changes in libstdc++, current root snapshots no longer
compile. root_v3.10.02 is the last release which can be build successfully.
6) The complex oop test included in oopack runs considerably slower when
compiled by gcc 3.4. It is 55.9 Mflops for gcc 3.3 but 40.7 Mflops for gcc
3.4. Additionally, there is still a substantial penalty for using oop methods
in contrast to c based ones (720 vs 40 Mflops).
  
Here are some of the results in detail:

qt-x11-free-3.3.0:
This is a problem with precompiled headers causing the C compiler to
crash. (See PR 14400)

pooma-gcc:
Since a week or so compiling most of the files included in the pooma-gcc
repository causes gcc to ice (see PR 14545).

Current boost cvs: 
Ten tests fail:
disjoint_sets/disjoint_set_test.cpp
graph/test/adj_list_cc.cpp
graph/test/subgraph.cpp
lambda/test/constructor_tests.cpp
lambda/test/operator_tests_simple.cpp
lambda/test/phoenix_control_structures.cpp
lambda/test/switch_construct.cpp
python/test/embedding.cpp
python/test/select_arg_to_python_test.cpp
thread/test/test_thread.cpp

Four of them are caused by recent changes in the libstdc++ implementation:
operator_tests_simple, phoenix_control_structures, switch_construct,
constructor_tests, all included in the lambda portion of boost, are rejected,
since the following code

 namespace std { 
 template <class T, class Allocator> class vector; 
} 
 
#include <vector> 
 using namespace std; 
 int main()  
{ 
    vector<int> v; 
} 

is rejected by gcc 3.4 but accepted by other compilers. I am not sure whether
Gaby's point of view (see PR 14370) is the way to proceed on this
subject. select_arg_to_python_test does not work because the linker cannot
find the symbol _Py_NoneStruct. This may be a local problem.

ACE-5.4+TAO-1.4+CIAO-0.4 :
The whole Ace, Tao and CIAO distribution compiles after applying some
patches. I ran all of the included tests which supplied an executable
run_test.pl. The Multicast_Test and the Proactor_Test of ACE fail. For the
tests Client_Leaks, Connection_Failure, BiDirectional_NestedUpcall,
Oneway_Buffering and Server_Leaks of TAO run_test did not return true.

root_v3.05.01:
Although gcc 3.4 compiles this code distribution, the stress test does not
pass. This is a regression with respect to gcc 3.3. All newer versions of root
do not compile. The file gcc3strm.cxx located in the root/cint/src directory
is rejected because of changes in the libstdc++ implementation.

gcc 3.4
OOPACK Version 1.7:
g++ -O3
Max=100000 Matrix=1000 Complex=100000 Iterator=100000

Max            100000    1.2   1.5   84.7  65.8    1.3
Matrix           1000    0.4   0.4  641.0 641.0    1.0
Complex        100000    1.1  19.7  720.7  40.7   17.7
Iterator       100000    0.3   0.3  689.7 645.2    1.1

gcc 3.3
OOPACK Version 1.7:
g++ -O3
Max=100000 Matrix=1000 Complex=100000 Iterator=100000

Max            100000    1.2   1.5   84.0  65.4    1.3
Matrix           1000    0.4   0.4  641.0 657.9    1.0
Complex        100000    1.1  14.3  733.9  55.9   13.1
Iterator       100000    0.3   0.3  666.7 666.7    1.0

FTensor--main--1.1pre25: No problems detected. All tests pass. On some
speed tests the performance of the optimizer is quite poor. Whereas
for the first, sixth, seventh, eighth and ninth tests the tensor based
implementation runs faster.

CXXFLAGS= -ftemplate-depth-1000 -Drestrict=__restrict__ -O3
-finline-functions -finline-limit-1000 -funroll-loops 

fast 1

real	0m0.901s
user	0m0.889s
sys	0m0.003s
Tensor 1

real	0m0.755s
user	0m0.742s
sys	0m0.006s
fast 2

real	0m1.956s
user	0m1.905s
sys	0m0.006s
Tensor 2

real	0m3.789s
user	0m3.103s
sys	0m0.021s
fast 3

real	0m0.456s
user	0m0.415s
sys	0m0.006s
Tensor 3

real	0m1.926s
user	0m1.896s
sys	0m0.006s
fast 4

real	0m0.644s
user	0m0.639s
sys	0m0.003s
Tensor 4

real	0m4.079s
user	0m4.007s
sys	0m0.018s
fast 5

real	0m0.962s
user	0m0.914s
sys	0m0.004s
Tensor 5

real	0m5.948s
user	0m5.646s
sys	0m0.024s
fast 6

real	0m6.680s
user	0m6.560s
sys	0m0.034s
Tensor 6

real	0m5.671s
user	0m5.640s
sys	0m0.023s
fast 7

real	0m13.301s
user	0m13.236s
sys	0m0.045s
Tensor 7

real	0m12.299s
user	0m12.245s
sys	0m0.039s
fast 8

real	0m28.533s
user	0m25.385s
sys	0m0.328s
Tensor 8

real	0m21.787s
user	0m20.313s
sys	0m0.218s
fast 9

real	0m37.726s
user	0m37.411s
sys	0m0.127s
Tensor 9

real	0m31.076s
user	0m30.941s
sys	0m0.101s


stepanov_v1p2.C: 

-O2
test      absolute   additions      ratio with
number    time       per second     test0

 0        0.14sec    357.14M         1.00
 1        0.14sec    357.14M         1.00
 2        0.16sec    312.50M         1.14
 3        0.14sec    357.14M         1.00
 4        0.14sec    357.14M         1.00
 5        0.15sec    333.33M         1.07
 6        0.14sec    357.14M         1.00
 7        0.14sec    357.14M         1.00
 8        0.15sec    333.33M         1.07
 9        0.15sec    333.33M         1.07
10        0.14sec    357.14M         1.00
11        0.14sec    357.14M         1.00
12        0.15sec    333.33M         1.07
mean:     0.14sec    346.07M         1.03

Total absolute time: 1.88 sec

Abstraction Penalty: 1.03

-O3

test      absolute   additions      ratio with
number    time       per second     test0

 0        0.14sec    357.14M         1.00
 1        0.14sec    357.14M         1.00
 2        0.14sec    357.14M         1.00
 3        0.14sec    357.14M         1.00
 4        0.14sec    357.14M         1.00
 5        0.15sec    333.33M         1.07
 6        0.14sec    357.14M         1.00
 7        0.14sec    357.14M         1.00
 8        0.14sec    357.14M         1.00
 9        0.14sec    357.14M         1.00
10        0.14sec    357.14M         1.00
11        0.14sec    357.14M         1.00
12        0.15sec    333.33M         1.07
mean:     0.14sec    353.37M         1.01

Total absolute time: 1.84 sec

Abstraction Penalty: 1.01


System setup:
SuSE 9
Glibc 2.3.2 
Linux 2.4.21
GNU ld version 2.14.90.0.5
g++ -v
Reading specs from /usr/local/lib/gcc/i686-pc-linux-gnu/3.4.0/specs
Configured with: ../gcc/configure --enable-threads=posix --enable-languages=c,c++,f77,objc --enable-__cxa_atexit --enable-libstdcxx-debug
Thread model: posix
gcc version 3.4.0 20040313 (prerelease)
Hope this helps,

Peter Schmid

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Paolo Carlini]

Thanks Peter,

>root_v3.05.01:
>Although gcc 3.4 compiles this code distribution, the stress test does not
>pass. This is a regression with respect to gcc 3.3. All newer versions of root
>do not compile. The file gcc3strm.cxx located in the root/cint/src directory
>is rejected because of changes in the libstdc++ implementation.
>  
>
I have just checked the latter issue (with root_v3.10.02): it's due 
streamoff being now
a 64 bit type (~long long), instead of a long. There isn't much we can 
do for this, if we
don't want to renounce to LFS support.

Changing gcc3strm.cxx to use streamoff (not an 'hardwired' long) and 
adding to
cint/lib/gcc3strm/iostrm.h something like:

    inline ostream& operator<< (ostream& ost, long long c)
      {return(ost.operator<<(c));}

and so on, should suffice to update the application for 3.4.

Paolo.

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Paolo Carlini]

poschmid@lbl.gov wrote:

>6) The complex oop test included in oopack runs considerably slower when
>compiled by gcc 3.4. It is 55.9 Mflops for gcc 3.3 but 40.7 Mflops for gcc
>3.4. Additionally, there is still a substantial penalty for using oop methods
>in contrast to c based ones (720 vs 40 Mflops).
>
Unfortunately, this is a long standing issue:

    http://gcc.gnu.org/ml/gcc/2003-01/msg01325.html

Which, however, is now fixed in the ssa-branch:

    http://gcc.gnu.org/ml/gcc/2004-01/msg01161.html

Seems unlikely that will be fixed in 3.4.x too... :(

Paolo.

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Giovanni Bajo]

Paolo Carlini wrote:

> Unfortunately, this is a long standing issue:
>
>     http://gcc.gnu.org/ml/gcc/2003-01/msg01325.html
>
> Which, however, is now fixed in the ssa-branch:
>
>     http://gcc.gnu.org/ml/gcc/2004-01/msg01161.html
>
> Seems unlikely that will be fixed in 3.4.x too... :(

Well, it looks like a serious regression... is there an open bug report with an
analysys/reduction for it? Even if it's too late for 3.4.x, maybe something can
be done for 3.4.1...

Giovanni Bajo

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Paolo Carlini]

Giovanni Bajo wrote:

>Well, it looks like a serious regression...
>
Yes :(

> is there an open bug report with an
>analysys/reduction for it?
>
I don't think there is one. Something I noticed 1 year ago is that in 
general P4 is
doing much worse than my old PII on this test: perhaps an hint for 
someone...

If nodoby more knowledgeable than me volunteer to prepare a detailed PR 
I will
before the end of this week.

> Even if it's too late for 3.4.x, maybe something can
>be done for 3.4.1...
>  
>
Agreed.

Paolo.

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Mark Mitchell]

poschmid@lbl.gov wrote:

>I ran many tests on my i686-pc-linux-gnu computer, the CPU is
>a Pentium IV with a 1.8 GHz clock. 
>  
>
Thanks for doing these tests.

>I tested the following applications: Blitz++-0.7, FTensor-1.1pre25,
>STLport-4.6.1, STLport-5.0-0409, qt-x11-free-3.3.0, root_v3.05.01,
>root_v3.10.02, root_v4.00.02, cln-1.1.5, ACE-5.4+TAO-1.4+CIAO-0.4, stepanov,
>oopack, mtl-2.1.2-21, estl, bench++, boost, the code examples that are
>included in the books by Breymann, Josuttis and Vandevoorde and other 
>applications.
>
>After applying some patches of my own, the remaining problems are:
>
>1)qt-x11-free-3.3.0 does not build (see PR 14400). This problem can be worked
>around by configuring with -system-zlib -system-libpng enabled. Although the
>build completes when configured this way, many applications crash on shutdown
>with a segmentation fault.
>  
>
Hmm.  That's an interesting PR.  I've retargeted it for 3.4.0.  I 
suspect that there is a bug in the PCH implementation, but it's hard to 
say until we can reproduce it more effectively.

>3) There are 10 failures while running the current boost regression test
>suite. All of them might be regressions since gcc 3.3 accepts the code which
>is rejected by gcc 3.4.  4) Some test examples of ACE and TAO fail.  
>5) Because of recent changes in libstdc++, current root snapshots no longer
>compile. root_v3.10.02 is the last release which can be build successfully.
>  
>
Until/unless you (or someone else!) does the analysis to figure out why 
these tests are failing it's hard to say what these problems mean.  The 
changes to G++ and V3 will undoubtedly make the compiler much stricter.

>6) The complex oop test included in oopack runs considerably slower when
>compiled by gcc 3.4. It is 55.9 Mflops for gcc 3.3 but 40.7 Mflops for gcc
>3.4. Additionally, there is still a substantial penalty for using oop methods
>in contrast to c based ones (720 vs 40 Mflops).
>  
>
I'll be taking a look at some of these problems over the next few days.

Thanks,

-- 
Mark Mitchell
CodeSourcery, LLC
(916) 791-8304
mark@codesourcery.com

Re: Results for g++ 3.4.0 application testing on i686-pc-linux-gnu

[Andrew Pinski]

On Mar 16, 2004, at 21:53, Mark Mitchell wrote:

>
>> 3) There are 10 failures while running the current boost regression 
>> test
>> suite. All of them might be regressions since gcc 3.3 accepts the 
>> code which
>> is rejected by gcc 3.4.  4) Some test examples of ACE and TAO fail.  
>> 5) Because of recent changes in libstdc++, current root snapshots no 
>> longer
>> compile. root_v3.10.02 is the last release which can be build 
>> successfully.
>>
> Until/unless you (or someone else!) does the analysis to figure out 
> why these tests are failing it's hard to say what these problems mean. 
>  The changes to G++ and V3 will undoubtedly make the compiler much 
> stricter.


The tests for ACE and TAO were filed and some of them have been fixed, 
I think the last
remaining one to be fixed is PR 14545.
I have not seen any libstdc++ regressions filed lately.

Thanks,
Andrew Pinski